2024-03-24T02:38:11.759 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-24T02:38:11.759 DEBUG:teuthology.run:Teuthology command: teuthology --owner mchangir --name mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi --archive /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480 --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} -- /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480/orig.config.yaml 2024-03-24T02:38:11.820 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-24T02:38:11.911 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480 branch: wip-mchangir-testing-main-20240318.032620 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} email: mchangir@redhat.com first_in_suite: false job_id: '7616480' kernel: client: sha1: distro flavor: default kdb: true sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-main-20240318.032620 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-main-20240318.032620 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 owner: mchangir priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3550 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 sleep_before_teardown: 0 subset: 613/1024 suite: fs suite_branch: wip-mchangir-testing-main-20240318.032620 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi017.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFUe7d7XjiupzXqAE5HnT+Gg5TRAtRnQbRE2JxfXVp+r309vjJvzhPnBsire81OyaoHVhH6cONnoY0+1PaOkIMs= smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIFq+/J9qEtq5l/LPnU8IS5rsliRZ1DBx8n7RblI04WyROxl+uPuIOsSgWhkwuIg2GFRANkhCEJd+GTScji7TUQ= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBon3nCu8YzMfhuhPKyUB3z6c+xnfjg13+X6+0I//2ZJIam9fr4+9G/H+hKTQnfyIv2KEuwGI1oHWXJeksZa2wg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_09:49:57 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-24T02:38:11.911 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa; will attempt to use it 2024-03-24T02:38:11.912 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks 2024-03-24T02:38:11.912 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-24T02:38:11.914 INFO:teuthology.task.internal:Checking packages... 2024-03-24T02:38:11.935 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3be9a5c9bc793e11e2800a8c0c696e8b46742033' 2024-03-24T02:38:11.935 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T02:38:11.935 INFO:teuthology.packaging:ref: None 2024-03-24T02:38:11.936 INFO:teuthology.packaging:tag: None 2024-03-24T02:38:11.936 INFO:teuthology.packaging:branch: wip-mchangir-testing-main-20240318.032620 2024-03-24T02:38:11.936 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T02:38:11.936 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-main-20240318.032620 2024-03-24T02:38:12.084 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2145.g3be9a5c9 2024-03-24T02:38:12.086 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-24T02:38:12.093 INFO:teuthology.task.internal:no buildpackages task found 2024-03-24T02:38:12.093 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-24T02:38:12.131 INFO:teuthology.task.internal:Saving configuration 2024-03-24T02:38:12.152 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-24T02:38:12.158 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T02:38:12.181 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 02:33:08.398239', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFUe7d7XjiupzXqAE5HnT+Gg5TRAtRnQbRE2JxfXVp+r309vjJvzhPnBsire81OyaoHVhH6cONnoY0+1PaOkIMs='} 2024-03-24T02:38:12.202 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 02:33:08.396749', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIFq+/J9qEtq5l/LPnU8IS5rsliRZ1DBx8n7RblI04WyROxl+uPuIOsSgWhkwuIg2GFRANkhCEJd+GTScji7TUQ='} 2024-03-24T02:38:12.226 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 02:33:08.399343', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBon3nCu8YzMfhuhPKyUB3z6c+xnfjg13+X6+0I//2ZJIam9fr4+9G/H+hKTQnfyIv2KEuwGI1oHWXJeksZa2wg='} 2024-03-24T02:38:12.226 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-24T02:38:12.232 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-24T02:38:12.232 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-24T02:38:12.232 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-24T02:38:12.232 INFO:teuthology.run_tasks:Running task console_log... 2024-03-24T02:38:12.348 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2b306810d0>, signals=[15]) 2024-03-24T02:38:12.349 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-24T02:38:12.355 INFO:teuthology.task.internal:Opening connections... 2024-03-24T02:38:12.355 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi017.front.sepia.ceph.com 2024-03-24T02:38:12.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T02:38:12.463 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-03-24T02:38:12.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T02:38:12.565 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-03-24T02:38:12.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T02:38:12.652 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-24T02:38:12.658 DEBUG:teuthology.orchestra.run.smithi017:> uname -m 2024-03-24T02:38:12.677 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2024-03-24T02:38:12.677 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2024-03-24T02:38:12.734 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="CentOS Stream" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="9" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:ID="centos" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="rhel fedora" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="9" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el9" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:LOGO="fedora-logo-icon" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://centos.org/" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-24T02:38:12.735 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-24T02:38:12.736 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2024-03-24T02:38:12.753 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-03-24T02:38:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-03-24T02:38:12.772 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-03-24T02:38:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-03-24T02:38:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-03-24T02:38:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-03-24T02:38:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-03-24T02:38:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-03-24T02:38:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-03-24T02:38:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-24T02:38:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-03-24T02:38:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-03-24T02:38:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-24T02:38:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-03-24T02:38:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-24T02:38:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-24T02:38:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-24T02:38:12.830 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-03-24T02:38:12.853 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-03-24T02:38:12.870 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-03-24T02:38:12.870 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-03-24T02:38:12.927 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-03-24T02:38:12.927 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-03-24T02:38:12.927 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-03-24T02:38:12.927 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-03-24T02:38:12.927 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-03-24T02:38:12.927 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-03-24T02:38:12.927 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-24T02:38:12.927 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-03-24T02:38:12.927 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-03-24T02:38:12.928 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-24T02:38:12.928 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-03-24T02:38:12.928 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-24T02:38:12.928 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-24T02:38:12.928 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-24T02:38:12.928 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-03-24T02:38:12.953 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-24T02:38:12.963 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-24T02:38:12.981 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-24T02:38:12.981 DEBUG:teuthology.orchestra.run.smithi017:> test '!' -e /home/ubuntu/cephtest 2024-03-24T02:38:12.983 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-03-24T02:38:12.985 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-03-24T02:38:13.001 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-24T02:38:13.006 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-24T02:38:13.006 DEBUG:teuthology.orchestra.run.smithi017:> test -z $(ls -A /var/lib/ceph) 2024-03-24T02:38:13.041 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-03-24T02:38:13.043 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-03-24T02:38:13.073 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-24T02:38:13.151 INFO:teuthology.run_tasks:Running task kernel... 2024-03-24T02:38:13.167 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'} 2024-03-24T02:38:13.168 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-24T02:38:13.168 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-24T02:38:13.168 DEBUG:teuthology.orchestra.run.smithi017:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-24T02:38:13.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T02:38:13.185 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-03-24T02:38:13.243 INFO:teuthology.orchestra.run.smithi017.stdout:5.14.0-361.el9.x86_64 2024-03-24T02:38:13.244 INFO:teuthology.task.kernel:Running kernel on smithi017: 5.14.0-361.el9.x86_64 2024-03-24T02:38:13.244 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum install -y kernel 2024-03-24T02:38:15.473 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-24T02:38:17.049 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-9 - BaseOS 4.9 MB/s | 2.0 MB 00:00 2024-03-24T02:38:26.513 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - BaseOS 902 kB/s | 8.0 MB 00:09 2024-03-24T02:38:29.364 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-03-24T02:38:33.607 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.1 MB 00:00 2024-03-24T02:38:35.124 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-03-24T02:38:36.628 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-24T02:38:41.859 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-03-24T02:38:44.008 INFO:teuthology.orchestra.run.smithi017.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-24T02:38:44.008 INFO:teuthology.orchestra.run.smithi017.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-24T02:38:44.104 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-03-24T02:38:44.109 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-03-24T02:38:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: Package Architecture Version Repository Size 2024-03-24T02:38:44.110 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-03-24T02:38:44.110 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-03-24T02:38:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-24T02:38:44.110 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-03-24T02:38:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-24T02:38:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-24T02:38:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-24T02:38:44.110 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:38:44.110 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-03-24T02:38:44.110 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-03-24T02:38:44.110 INFO:teuthology.orchestra.run.smithi017.stdout:Install 4 Packages 2024-03-24T02:38:44.111 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:38:44.111 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 76 M 2024-03-24T02:38:44.111 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 123 M 2024-03-24T02:38:44.111 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-03-24T02:38:44.769 INFO:teuthology.orchestra.run.smithi017.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 72 kB/s | 44 kB 00:00 2024-03-24T02:39:03.568 INFO:teuthology.orchestra.run.smithi017.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 786 kB/s | 15 MB 00:19 2024-03-24T02:39:20.333 INFO:teuthology.orchestra.run.smithi017.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 791 kB/s | 27 MB 00:35 2024-03-24T02:39:27.441 INFO:teuthology.orchestra.run.smithi017.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 791 kB/s | 33 MB 00:43 2024-03-24T02:39:27.441 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-03-24T02:39:27.442 INFO:teuthology.orchestra.run.smithi017.stdout:Total 1.8 MB/s | 76 MB 00:43 2024-03-24T02:39:28.134 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-03-24T02:39:28.571 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-03-24T02:39:28.572 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-03-24T02:39:29.610 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-03-24T02:39:29.610 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-03-24T02:39:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-03-24T02:39:32.878 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-24T02:39:33.032 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T02:39:35.914 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T02:39:36.051 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T02:39:41.155 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T02:39:41.193 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T02:39:46.319 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T02:40:16.998 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T02:40:16.998 INFO:teuthology.orchestra.run.smithi017.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-24T02:40:16.998 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:40:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-24T02:40:17.426 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T02:40:17.426 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-24T02:40:17.426 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T02:40:17.426 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T02:40:17.854 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T02:40:17.854 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:40:17.854 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-03-24T02:40:17.854 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-24T02:40:17.854 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-24T02:40:17.855 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-24T02:40:17.855 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-24T02:40:17.855 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:40:17.855 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-03-24T02:40:18.225 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T02:40:18.327 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T02:40:18.327 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T02:40:18.327 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-24T02:40:18.327 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-24T02:40:18.327 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-24T02:40:18.328 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-24T02:40:18.328 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi017.front.sepia.ceph.com, path=None, version=distro) 2024-03-24T02:40:18.328 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T02:40:18.411 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T02:40:18.411 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T02:40:18.412 DEBUG:teuthology.orchestra.run.smithi017:> sudo rpm -qi grub2-tools 2024-03-24T02:40:18.450 INFO:teuthology.orchestra.run.smithi017.stdout:Name : grub2-tools 2024-03-24T02:40:18.450 INFO:teuthology.orchestra.run.smithi017.stdout:Epoch : 1 2024-03-24T02:40:18.450 INFO:teuthology.orchestra.run.smithi017.stdout:Version : 2.06 2024-03-24T02:40:18.450 INFO:teuthology.orchestra.run.smithi017.stdout:Release : 68.el9 2024-03-24T02:40:18.450 INFO:teuthology.orchestra.run.smithi017.stdout:Architecture: x86_64 2024-03-24T02:40:18.450 INFO:teuthology.orchestra.run.smithi017.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:Group : Unspecified 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:Size : 8264254 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:License : GPLv3+ 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:Packager : builder@centos.org 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:Vendor : CentOS 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:Summary : Support tools for GRUB. 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:Description : 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-24T02:40:18.451 INFO:teuthology.orchestra.run.smithi017.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-24T02:40:18.452 INFO:teuthology.orchestra.run.smithi017.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-24T02:40:18.452 INFO:teuthology.orchestra.run.smithi017.stdout:hardware devices. 2024-03-24T02:40:18.452 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:40:18.452 INFO:teuthology.orchestra.run.smithi017.stdout:This subpackage provides tools for support of all platforms. 2024-03-24T02:40:18.453 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-24T02:40:18.453 INFO:teuthology.task.kernel:Updating grub on smithi017 to boot 5.14.0-430.el9.x86_64 2024-03-24T02:40:18.453 DEBUG:teuthology.orchestra.run.smithi017:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-24T02:40:19.122 INFO:teuthology.orchestra.run.smithi017.stderr:Generating grub configuration file ... 2024-03-24T02:40:20.594 INFO:teuthology.orchestra.run.smithi017.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-24T02:40:20.620 INFO:teuthology.orchestra.run.smithi017.stderr:done 2024-03-24T02:40:20.622 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-03-24T02:40:20.638 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.72USvG5oBA 2024-03-24T02:40:20.638 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.72USvG5oBA 2024-03-24T02:40:20.713 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.72USvG5oBA 2024-03-24T02:40:20.857 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.72USvG5oBA is 6KB 2024-03-24T02:40:20.906 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.72USvG5oBA 2024-03-24T02:40:20.920 DEBUG:teuthology.orchestra.run.smithi017:> sudo /bin/ls /boot/loader/entries || true 2024-03-24T02:40:20.989 INFO:teuthology.orchestra.run.smithi017.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-24T02:40:20.990 INFO:teuthology.orchestra.run.smithi017.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-24T02:40:20.990 INFO:teuthology.orchestra.run.smithi017.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-24T02:40:20.990 INFO:teuthology.orchestra.run.smithi017.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-24T02:40:20.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-24T02:40:21.149 DEBUG:teuthology.orchestra.run.smithi017:> sudo shutdown -r now 2024-03-24T02:40:51.152 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-24T02:40:51.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-24T02:40:51.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T02:41:09.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.17 2024-03-24T02:41:18.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-24T02:41:18.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T02:42:18.667 DEBUG:teuthology.orchestra.remote:timed out 2024-03-24T02:42:30.668 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-24T02:42:30.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T02:42:31.056 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-03-24T02:42:31.454 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-24T02:42:31.454 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-430.el9.x86_64"... 2024-03-24T02:42:31.454 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-03-24T02:42:31.470 INFO:teuthology.orchestra.run.smithi017.stdout:5.14.0-430.el9.x86_64 2024-03-24T02:42:31.471 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-24T02:42:31.471 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-24T02:42:31.471 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-24T02:42:32.472 DEBUG:teuthology.parallel:result is None 2024-03-24T02:42:32.472 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-24T02:42:32.480 INFO:teuthology.task.internal:Creating test directory... 2024-03-24T02:42:32.480 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-24T02:42:32.482 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-24T02:42:32.485 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-24T02:42:32.503 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-24T02:42:32.526 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-24T02:42:32.560 INFO:teuthology.task.internal:Creating archive directory... 2024-03-24T02:42:32.560 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-24T02:42:32.562 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-24T02:42:32.565 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-24T02:42:32.680 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-24T02:42:32.686 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-24T02:42:32.686 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-24T02:42:32.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T02:42:32.704 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-24T02:42:32.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T02:42:32.720 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-24T02:42:32.722 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-24T02:42:32.747 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-24T02:42:32.790 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T02:42:32.811 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T02:42:32.813 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T02:42:32.851 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T02:42:32.854 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T02:42:32.872 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T02:42:32.874 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-24T02:42:32.881 INFO:teuthology.task.internal:Configuring sudo... 2024-03-24T02:42:32.881 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-24T02:42:32.883 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-24T02:42:32.885 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-24T02:42:32.917 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-24T02:42:32.926 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-24T02:42:32.926 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-24T02:42:32.954 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-24T02:42:32.960 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-24T02:42:32.975 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T02:42:33.052 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T02:42:33.100 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T02:42:33.182 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T02:42:33.231 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:42:33.231 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-24T02:42:33.303 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T02:42:33.345 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T02:42:33.412 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T02:42:33.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T02:42:33.503 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-24T02:42:33.503 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-24T02:42:33.576 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T02:42:33.620 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T02:42:33.680 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T02:42:33.722 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T02:42:33.766 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-24T02:42:33.766 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-24T02:42:33.834 DEBUG:teuthology.orchestra.run.smithi017:> sudo service rsyslog restart 2024-03-24T02:42:33.836 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-03-24T02:42:33.838 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-03-24T02:42:33.882 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T02:42:33.884 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T02:42:33.911 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T02:42:34.401 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-24T02:42:34.408 INFO:teuthology.task.internal:Starting timer... 2024-03-24T02:42:34.408 INFO:teuthology.run_tasks:Running task pcp... 2024-03-24T02:42:34.446 INFO:teuthology.run_tasks:Running task selinux... 2024-03-24T02:42:34.477 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-24T02:42:34.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo service auditd rotate 2024-03-24T02:42:34.524 INFO:teuthology.orchestra.run.smithi017.stdout:Rotating logs: 2024-03-24T02:42:34.526 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-03-24T02:42:34.583 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-03-24T02:42:34.585 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-03-24T02:42:34.632 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-03-24T02:42:34.634 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-24T02:42:34.634 DEBUG:teuthology.orchestra.run.smithi017:> /usr/sbin/getenforce 2024-03-24T02:42:34.659 INFO:teuthology.orchestra.run.smithi017.stdout:Permissive 2024-03-24T02:42:34.659 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-03-24T02:42:34.681 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-03-24T02:42:34.681 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-03-24T02:42:34.705 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-03-24T02:42:34.705 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi017.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-03-24T02:42:34.705 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T02:42:34.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T02:42:34.734 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T02:42:34.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T02:42:34.767 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T02:42:34.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T02:42:34.794 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-24T02:42:34.794 DEBUG:teuthology.orchestra.run.smithi017:> sudo /usr/sbin/setenforce permissive 2024-03-24T02:42:34.827 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-03-24T02:42:34.866 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-03-24T02:42:34.898 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-24T02:42:34.907 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-24T02:42:34.912 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-24T02:42:35.086 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-24T02:42:35.104 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-24T02:42:35.106 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi017.front.sepia.ceph.com,smithi019.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-24T02:47:18.814 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi017.front.sepia.ceph.com'), Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-03-24T02:47:18.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-24T02:47:18.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T02:47:18.907 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-03-24T02:47:18.985 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-24T02:47:18.985 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-24T02:47:18.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T02:47:19.085 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-03-24T02:47:19.157 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-24T02:47:19.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-03-24T02:47:19.158 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T02:47:19.245 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-03-24T02:47:19.313 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-03-24T02:47:19.314 INFO:teuthology.run_tasks:Running task clock... 2024-03-24T02:47:19.323 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-24T02:47:19.324 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-24T02:47:19.324 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T02:47:19.326 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-24T02:47:19.327 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T02:47:19.329 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-24T02:47:19.329 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T02:47:19.359 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-24T02:47:19.361 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-24T02:47:19.377 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-24T02:47:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-24T02:47:19.386 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-24T02:47:19.403 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-24T02:47:19.409 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: ntpd: command not found 2024-03-24T02:47:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-03-24T02:47:19.423 INFO:teuthology.orchestra.run.smithi017.stdout:506 Cannot talk to daemon 2024-03-24T02:47:19.432 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-03-24T02:47:19.434 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-03-24T02:47:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-24T02:47:19.446 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-03-24T02:47:19.454 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-24T02:47:19.457 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-24T02:47:19.462 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-24T02:47:19.474 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-24T02:47:19.478 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-24T02:47:19.511 INFO:teuthology.orchestra.run.smithi017.stderr:bash: line 1: ntpq: command not found 2024-03-24T02:47:19.515 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T02:47:19.515 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2024-03-24T02:47:19.515 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T02:47:19.515 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T02:47:19.516 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T02:47:19.516 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T02:47:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-03-24T02:47:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-03-24T02:47:19.525 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T02:47:19.525 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-03-24T02:47:19.525 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T02:47:19.525 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T02:47:19.526 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T02:47:19.526 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T02:47:19.526 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T02:47:19.527 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-03-24T02:47:19.527 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T02:47:19.527 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T02:47:19.527 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T02:47:19.527 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T02:47:19.527 INFO:teuthology.run_tasks:Running task install... 2024-03-24T02:47:19.535 DEBUG:teuthology.task.install:project ceph 2024-03-24T02:47:19.535 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}} 2024-03-24T02:47:19.535 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-24T02:47:19.535 INFO:teuthology.task.install:Using flavor: default 2024-03-24T02:47:19.541 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-24T02:47:19.541 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-24T02:47:19.541 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-24T02:47:19.541 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T02:47:19.543 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-24T02:47:19.544 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T02:47:19.544 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-24T02:47:19.545 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T02:47:19.694 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-24T02:47:19.694 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-24T02:47:19.762 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-24T02:47:19.762 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-24T02:47:19.765 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-24T02:47:19.765 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-24T02:47:19.802 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-24T02:47:19.802 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:47:19.802 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-24T02:47:19.831 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-24T02:47:19.831 DEBUG:teuthology.orchestra.run.smithi017:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-24T02:47:19.868 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-24T02:47:19.868 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-24T02:47:19.868 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-24T02:47:19.899 DEBUG:teuthology.orchestra.run.smithi017:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-24T02:47:19.900 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-24T02:47:19.900 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-24T02:47:19.930 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-24T02:47:19.931 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-24T02:47:19.931 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-24T02:47:19.959 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-24T02:47:19.959 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-24T02:47:19.972 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-24T02:47:19.978 DEBUG:teuthology.orchestra.run.smithi017:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-24T02:47:20.027 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-24T02:47:20.045 INFO:teuthology.orchestra.run.smithi017.stdout:check_obsoletes = 1 2024-03-24T02:47:20.046 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum clean all 2024-03-24T02:47:20.059 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-24T02:47:20.088 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-03-24T02:47:20.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-03-24T02:47:20.107 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-24T02:47:20.173 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-03-24T02:47:20.175 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-03-24T02:47:20.305 INFO:teuthology.orchestra.run.smithi017.stdout:74 files removed 2024-03-24T02:47:20.332 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-24T02:47:20.337 INFO:teuthology.orchestra.run.smithi019.stdout:74 files removed 2024-03-24T02:47:20.365 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-24T02:47:20.410 INFO:teuthology.orchestra.run.smithi188.stdout:74 files removed 2024-03-24T02:47:20.437 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-24T02:47:21.104 INFO:teuthology.orchestra.run.smithi017.stdout:ceph packages for x86_64 178 kB/s | 83 kB 00:00 2024-03-24T02:47:21.132 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 180 kB/s | 83 kB 00:00 2024-03-24T02:47:21.150 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 197 kB/s | 83 kB 00:00 2024-03-24T02:47:21.471 INFO:teuthology.orchestra.run.smithi017.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-24T02:47:21.499 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 31 kB/s | 10 kB 00:00 2024-03-24T02:47:21.500 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-24T02:47:21.841 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-03-24T02:47:21.847 INFO:teuthology.orchestra.run.smithi017.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-03-24T02:47:21.859 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-03-24T02:47:22.319 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-24T02:47:22.345 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-24T02:47:22.549 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-24T02:47:23.827 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - BaseOS 5.6 MB/s | 2.0 MB 00:00 2024-03-24T02:47:24.008 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-03-24T02:47:24.012 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-03-24T02:47:25.503 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.0 MB 00:01 2024-03-24T02:47:40.631 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - BaseOS 505 kB/s | 8.0 MB 00:16 2024-03-24T02:47:41.225 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 482 kB/s | 8.0 MB 00:17 2024-03-24T02:47:43.110 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-03-24T02:47:43.584 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-03-24T02:47:47.563 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.1 MB 00:00 2024-03-24T02:47:49.512 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-03-24T02:47:50.031 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - AppStream 2.3 MB/s | 19 MB 00:08 2024-03-24T02:47:51.674 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-03-24T02:47:54.557 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.1 MB 00:00 2024-03-24T02:47:55.266 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 764 kB/s | 6.1 MB 00:08 2024-03-24T02:47:56.049 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-03-24T02:47:56.725 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-03-24T02:47:56.788 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-03-24T02:47:57.017 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-03-24T02:47:59.012 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 21 MB 00:02 2024-03-24T02:47:59.244 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-24T02:47:59.245 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-24T02:47:59.260 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-24T02:47:59.261 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-24T02:47:59.262 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-24T02:47:59.263 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-24T02:47:59.265 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-24T02:47:59.268 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-24T02:47:59.269 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-24T02:47:59.354 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-24T02:47:59.382 INFO:teuthology.orchestra.run.smithi019.stdout:============================================================================================ 2024-03-24T02:47:59.382 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-03-24T02:47:59.382 INFO:teuthology.orchestra.run.smithi019.stdout:============================================================================================ 2024-03-24T02:47:59.382 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-24T02:47:59.382 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-24T02:47:59.382 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-24T02:47:59.383 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-24T02:47:59.384 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-24T02:47:59.385 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-24T02:47:59.386 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-24T02:47:59.386 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-24T02:47:59.386 INFO:teuthology.orchestra.run.smithi019.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-24T02:47:59.386 INFO:teuthology.orchestra.run.smithi019.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-24T02:47:59.386 INFO:teuthology.orchestra.run.smithi019.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-24T02:47:59.386 INFO:teuthology.orchestra.run.smithi019.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-24T02:47:59.386 INFO:teuthology.orchestra.run.smithi019.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-24T02:47:59.386 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-24T02:47:59.386 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-24T02:47:59.386 INFO:teuthology.orchestra.run.smithi019.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-24T02:47:59.386 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-24T02:47:59.386 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-24T02:47:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-24T02:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-24T02:47:59.389 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-24T02:47:59.390 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-24T02:47:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-24T02:47:59.392 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-24T02:47:59.393 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-24T02:47:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-24T02:47:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-24T02:47:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-24T02:47:59.395 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-03-24T02:47:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-24T02:47:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:47:59.395 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-24T02:47:59.395 INFO:teuthology.orchestra.run.smithi019.stdout:============================================================================================ 2024-03-24T02:47:59.395 INFO:teuthology.orchestra.run.smithi019.stdout:Install 150 Packages 2024-03-24T02:47:59.395 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 24 Packages 2024-03-24T02:47:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:47:59.395 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 224 M 2024-03-24T02:47:59.396 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-24T02:47:59.848 INFO:teuthology.orchestra.run.smithi019.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-03-24T02:48:00.140 INFO:teuthology.orchestra.run.smithi019.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 3.0 MB/s | 890 kB 00:00 2024-03-24T02:48:00.223 INFO:teuthology.orchestra.run.smithi019.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 1.7 MB/s | 145 kB 00:00 2024-03-24T02:48:00.482 INFO:teuthology.orchestra.run.smithi019.stdout:(4/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 6.4 MB/s | 5.4 MB 00:00 2024-03-24T02:48:00.674 INFO:teuthology.orchestra.run.smithi019.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-03-24T02:48:00.716 INFO:teuthology.orchestra.run.smithi019.stdout:(6/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 4.7 MB/s | 2.3 MB 00:00 2024-03-24T02:48:01.208 INFO:teuthology.orchestra.run.smithi019.stdout:(7/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 8.8 MB/s | 4.7 MB 00:00 2024-03-24T02:48:01.658 INFO:teuthology.orchestra.run.smithi019.stdout:(8/174): ceph-common-19.0.0-2145.g3be9a5c9.el9. 10 MB/s | 20 MB 00:02 2024-03-24T02:48:01.733 INFO:teuthology.orchestra.run.smithi019.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 337 kB/s | 25 kB 00:00 2024-03-24T02:48:02.076 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-24T02:48:02.159 INFO:teuthology.orchestra.run.smithi019.stdout:(10/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 10 MB/s | 9.9 MB 00:00 2024-03-24T02:48:02.250 INFO:teuthology.orchestra.run.smithi019.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 2.4 MB/s | 220 kB 00:00 2024-03-24T02:48:02.326 INFO:teuthology.orchestra.run.smithi019.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 360 kB/s | 27 kB 00:00 2024-03-24T02:48:02.401 INFO:teuthology.orchestra.run.smithi019.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 417 kB/s | 31 kB 00:00 2024-03-24T02:48:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 6.7 MB/s | 746 kB 00:00 2024-03-24T02:48:02.602 INFO:teuthology.orchestra.run.smithi019.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.7 MB/s | 163 kB 00:00 2024-03-24T02:48:02.685 INFO:teuthology.orchestra.run.smithi019.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.5 MB/s | 127 kB 00:00 2024-03-24T02:48:02.785 INFO:teuthology.orchestra.run.smithi019.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 4.9 MB/s | 501 kB 00:00 2024-03-24T02:48:03.369 INFO:teuthology.orchestra.run.smithi019.stdout:(18/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 8.8 MB/s | 5.1 MB 00:00 2024-03-24T02:48:03.536 INFO:teuthology.orchestra.run.smithi019.stdout:(19/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 6.0 MB/s | 17 MB 00:02 2024-03-24T02:48:03.561 INFO:teuthology.orchestra.run.smithi019.stdout:(20/174): python3-ceph-argparse-19.0.0-2145.g3b 235 kB/s | 45 kB 00:00 2024-03-24T02:48:03.595 INFO:teuthology.orchestra.run.smithi019.stdout:(21/174): python3-ceph-common-19.0.0-2145.g3be9 2.2 MB/s | 131 kB 00:00 2024-03-24T02:48:03.645 INFO:teuthology.orchestra.run.smithi019.stdout:(22/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 1.9 MB/s | 162 kB 00:00 2024-03-24T02:48:03.670 INFO:teuthology.orchestra.run.smithi019.stdout:(23/174): python3-rados-19.0.0-2145.g3be9a5c9.e 4.2 MB/s | 321 kB 00:00 2024-03-24T02:48:03.721 INFO:teuthology.orchestra.run.smithi019.stdout:(24/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 3.9 MB/s | 298 kB 00:00 2024-03-24T02:48:03.746 INFO:teuthology.orchestra.run.smithi019.stdout:(25/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.3 MB/s | 99 kB 00:00 2024-03-24T02:48:03.796 INFO:teuthology.orchestra.run.smithi019.stdout:(26/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-24T02:48:03.888 INFO:teuthology.orchestra.run.smithi019.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-24T02:48:03.980 INFO:teuthology.orchestra.run.smithi019.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 370 kB/s | 34 kB 00:00 2024-03-24T02:48:03.984 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-24T02:48:04.072 INFO:teuthology.orchestra.run.smithi019.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 1.5 MB/s | 139 kB 00:00 2024-03-24T02:48:04.248 INFO:teuthology.orchestra.run.smithi019.stdout:(30/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 6.2 MB/s | 3.1 MB 00:00 2024-03-24T02:48:04.289 INFO:teuthology.orchestra.run.smithi019.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 8.3 MB/s | 1.8 MB 00:00 2024-03-24T02:48:04.398 INFO:teuthology.orchestra.run.smithi019.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 2.2 MB/s | 248 kB 00:00 2024-03-24T02:48:04.490 INFO:teuthology.orchestra.run.smithi019.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 538 kB/s | 49 kB 00:00 2024-03-24T02:48:04.548 INFO:teuthology.orchestra.run.smithi017.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-24T02:48:04.550 INFO:teuthology.orchestra.run.smithi017.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-24T02:48:04.565 INFO:teuthology.orchestra.run.smithi017.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-24T02:48:04.566 INFO:teuthology.orchestra.run.smithi017.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-24T02:48:04.567 INFO:teuthology.orchestra.run.smithi017.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-24T02:48:04.568 INFO:teuthology.orchestra.run.smithi017.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-24T02:48:04.570 INFO:teuthology.orchestra.run.smithi017.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-24T02:48:04.573 INFO:teuthology.orchestra.run.smithi017.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-24T02:48:04.574 INFO:teuthology.orchestra.run.smithi017.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-24T02:48:04.582 INFO:teuthology.orchestra.run.smithi019.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 179 kB/s | 16 kB 00:00 2024-03-24T02:48:04.662 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-03-24T02:48:04.685 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================ 2024-03-24T02:48:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-03-24T02:48:04.685 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================ 2024-03-24T02:48:04.685 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-03-24T02:48:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-24T02:48:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-24T02:48:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-24T02:48:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-24T02:48:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-24T02:48:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-24T02:48:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-24T02:48:04.686 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-24T02:48:04.687 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrading: 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-24T02:48:04.688 INFO:teuthology.orchestra.run.smithi017.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-24T02:48:04.689 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-24T02:48:04.690 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-24T02:48:04.691 INFO:teuthology.orchestra.run.smithi017.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-24T02:48:04.692 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-24T02:48:04.693 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-24T02:48:04.694 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-24T02:48:04.695 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-24T02:48:04.696 INFO:teuthology.orchestra.run.smithi017.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-24T02:48:04.697 INFO:teuthology.orchestra.run.smithi017.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-24T02:48:04.697 INFO:teuthology.orchestra.run.smithi017.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-24T02:48:04.697 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-24T02:48:04.697 INFO:teuthology.orchestra.run.smithi017.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-24T02:48:04.697 INFO:teuthology.orchestra.run.smithi017.stdout:Installing weak dependencies: 2024-03-24T02:48:04.697 INFO:teuthology.orchestra.run.smithi017.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-24T02:48:04.697 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:48:04.697 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-03-24T02:48:04.697 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================ 2024-03-24T02:48:04.697 INFO:teuthology.orchestra.run.smithi017.stdout:Install 150 Packages 2024-03-24T02:48:04.697 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrade 24 Packages 2024-03-24T02:48:04.697 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:48:04.698 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 224 M 2024-03-24T02:48:04.698 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-03-24T02:48:05.024 INFO:teuthology.orchestra.run.smithi019.stdout:(35/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 14 MB/s | 48 MB 00:03 2024-03-24T02:48:05.049 INFO:teuthology.orchestra.run.smithi019.stdout:(36/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 565 kB/s | 264 kB 00:00 2024-03-24T02:48:05.116 INFO:teuthology.orchestra.run.smithi019.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 273 kB/s | 18 kB 00:00 2024-03-24T02:48:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:(38/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 6.3 MB/s | 748 kB 00:00 2024-03-24T02:48:05.292 INFO:teuthology.orchestra.run.smithi019.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 7.1 MB/s | 7.4 MB 00:01 2024-03-24T02:48:05.294 INFO:teuthology.orchestra.run.smithi017.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-03-24T02:48:05.359 INFO:teuthology.orchestra.run.smithi019.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 662 kB 00:00 2024-03-24T02:48:05.384 INFO:teuthology.orchestra.run.smithi019.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 106 kB 00:00 2024-03-24T02:48:05.409 INFO:teuthology.orchestra.run.smithi019.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 920 kB 00:00 2024-03-24T02:48:05.443 INFO:teuthology.orchestra.run.smithi019.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2024-03-24T02:48:05.468 INFO:teuthology.orchestra.run.smithi019.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 381 kB/s | 32 kB 00:00 2024-03-24T02:48:05.493 INFO:teuthology.orchestra.run.smithi019.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 343 kB/s | 17 kB 00:00 2024-03-24T02:48:05.536 INFO:teuthology.orchestra.run.smithi017.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 3.6 MB/s | 890 kB 00:00 2024-03-24T02:48:05.560 INFO:teuthology.orchestra.run.smithi019.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-24T02:48:05.603 INFO:teuthology.orchestra.run.smithi017.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 2.1 MB/s | 145 kB 00:00 2024-03-24T02:48:05.686 INFO:teuthology.orchestra.run.smithi017.stdout:(4/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 9.6 MB/s | 5.4 MB 00:00 2024-03-24T02:48:05.702 INFO:teuthology.orchestra.run.smithi019.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 43 kB/s | 10 kB 00:00 2024-03-24T02:48:05.727 INFO:teuthology.orchestra.run.smithi019.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 689 kB/s | 159 kB 00:00 2024-03-24T02:48:05.752 INFO:teuthology.orchestra.run.smithi019.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 230 kB/s | 44 kB 00:00 2024-03-24T02:48:05.778 INFO:teuthology.orchestra.run.smithi019.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-24T02:48:05.803 INFO:teuthology.orchestra.run.smithi019.stdout:(51/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-24T02:48:05.828 INFO:teuthology.orchestra.run.smithi019.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 616 kB/s | 46 kB 00:00 2024-03-24T02:48:05.829 INFO:teuthology.orchestra.run.smithi017.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 11 MB/s | 1.5 MB 00:00 2024-03-24T02:48:05.854 INFO:teuthology.orchestra.run.smithi019.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-24T02:48:05.879 INFO:teuthology.orchestra.run.smithi019.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-24T02:48:05.904 INFO:teuthology.orchestra.run.smithi019.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-24T02:48:05.928 INFO:teuthology.orchestra.run.smithi017.stdout:(6/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 7.1 MB/s | 2.3 MB 00:00 2024-03-24T02:48:05.929 INFO:teuthology.orchestra.run.smithi019.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 460 kB/s | 34 kB 00:00 2024-03-24T02:48:05.996 INFO:teuthology.orchestra.run.smithi019.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 17 MB/s | 1.5 MB 00:00 2024-03-24T02:48:06.071 INFO:teuthology.orchestra.run.smithi019.stdout:(58/174): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-03-24T02:48:06.105 INFO:teuthology.orchestra.run.smithi019.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 4.5 MB/s | 811 kB 00:00 2024-03-24T02:48:06.130 INFO:teuthology.orchestra.run.smithi019.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.0 MB/s | 134 kB 00:00 2024-03-24T02:48:06.137 INFO:teuthology.orchestra.run.smithi017.stdout:(7/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 15 MB/s | 4.7 MB 00:00 2024-03-24T02:48:06.256 INFO:teuthology.orchestra.run.smithi019.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2024-03-24T02:48:06.281 INFO:teuthology.orchestra.run.smithi019.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 527 kB/s | 78 kB 00:00 2024-03-24T02:48:06.315 INFO:teuthology.orchestra.run.smithi019.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.6 MB/s | 1.0 MB 00:00 2024-03-24T02:48:06.339 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-24T02:48:06.340 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-24T02:48:06.354 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-24T02:48:06.355 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-24T02:48:06.356 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-24T02:48:06.357 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-24T02:48:06.359 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-24T02:48:06.362 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-24T02:48:06.363 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-24T02:48:06.424 INFO:teuthology.orchestra.run.smithi019.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 35 MB/s | 5.8 MB 00:00 2024-03-24T02:48:06.443 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-03-24T02:48:06.449 INFO:teuthology.orchestra.run.smithi019.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 149 kB/s | 24 kB 00:00 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-24T02:48:06.456 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-24T02:48:06.457 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-24T02:48:06.458 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-24T02:48:06.459 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-24T02:48:06.460 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-24T02:48:06.461 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-24T02:48:06.462 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-24T02:48:06.462 INFO:teuthology.orchestra.run.smithi188.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-24T02:48:06.462 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-24T02:48:06.462 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-24T02:48:06.462 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-24T02:48:06.462 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-24T02:48:06.462 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-24T02:48:06.462 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-24T02:48:06.462 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-24T02:48:06.462 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-24T02:48:06.462 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-24T02:48:06.463 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-24T02:48:06.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-24T02:48:06.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-24T02:48:06.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-24T02:48:06.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-24T02:48:06.468 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-24T02:48:06.469 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-24T02:48:06.469 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-24T02:48:06.469 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-24T02:48:06.469 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2024-03-24T02:48:06.469 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-24T02:48:06.469 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:48:06.469 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-03-24T02:48:06.469 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-03-24T02:48:06.469 INFO:teuthology.orchestra.run.smithi188.stdout:Install 150 Packages 2024-03-24T02:48:06.469 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 24 Packages 2024-03-24T02:48:06.469 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:48:06.470 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 224 M 2024-03-24T02:48:06.470 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-03-24T02:48:06.474 INFO:teuthology.orchestra.run.smithi019.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 116 kB/s | 18 kB 00:00 2024-03-24T02:48:06.499 INFO:teuthology.orchestra.run.smithi019.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-24T02:48:06.524 INFO:teuthology.orchestra.run.smithi019.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-24T02:48:06.599 INFO:teuthology.orchestra.run.smithi019.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-24T02:48:06.625 INFO:teuthology.orchestra.run.smithi019.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-03-24T02:48:06.650 INFO:teuthology.orchestra.run.smithi019.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-03-24T02:48:06.675 INFO:teuthology.orchestra.run.smithi019.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 919 kB/s | 69 kB 00:00 2024-03-24T02:48:06.709 INFO:teuthology.orchestra.run.smithi019.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 702 kB/s | 41 kB 00:00 2024-03-24T02:48:06.734 INFO:teuthology.orchestra.run.smithi019.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-24T02:48:06.760 INFO:teuthology.orchestra.run.smithi019.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-03-24T02:48:06.785 INFO:teuthology.orchestra.run.smithi019.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-24T02:48:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 632 kB/s | 47 kB 00:00 2024-03-24T02:48:06.812 INFO:teuthology.orchestra.run.smithi017.stdout:(8/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el9 15 MB/s | 9.9 MB 00:00 2024-03-24T02:48:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 568 kB/s | 43 kB 00:00 2024-03-24T02:48:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 775 kB/s | 39 kB 00:00 2024-03-24T02:48:06.863 INFO:teuthology.orchestra.run.smithi017.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 505 kB/s | 25 kB 00:00 2024-03-24T02:48:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 491 kB/s | 41 kB 00:00 2024-03-24T02:48:06.995 INFO:teuthology.orchestra.run.smithi019.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 300 kB 00:00 2024-03-24T02:48:07.170 INFO:teuthology.orchestra.run.smithi019.stdout:(82/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-03-24T02:48:07.195 INFO:teuthology.orchestra.run.smithi019.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 207 kB/s | 56 kB 00:00 2024-03-24T02:48:07.220 INFO:teuthology.orchestra.run.smithi019.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 281 kB/s | 63 kB 00:00 2024-03-24T02:48:07.245 INFO:teuthology.orchestra.run.smithi019.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-03-24T02:48:07.271 INFO:teuthology.orchestra.run.smithi019.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-24T02:48:07.296 INFO:teuthology.orchestra.run.smithi019.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-24T02:48:07.329 INFO:teuthology.orchestra.run.smithi019.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-24T02:48:07.363 INFO:teuthology.orchestra.run.smithi019.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2024-03-24T02:48:07.388 INFO:teuthology.orchestra.run.smithi019.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-24T02:48:07.430 INFO:teuthology.orchestra.run.smithi019.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 187 kB 00:00 2024-03-24T02:48:07.547 INFO:teuthology.orchestra.run.smithi019.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-24T02:48:07.573 INFO:teuthology.orchestra.run.smithi019.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-24T02:48:07.598 INFO:teuthology.orchestra.run.smithi019.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-24T02:48:07.631 INFO:teuthology.orchestra.run.smithi019.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-03-24T02:48:07.913 INFO:teuthology.orchestra.run.smithi017.stdout:(10/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 8.5 MB/s | 17 MB 00:01 2024-03-24T02:48:07.951 INFO:teuthology.orchestra.run.smithi188.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-03-24T02:48:07.963 INFO:teuthology.orchestra.run.smithi017.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 4.3 MB/s | 220 kB 00:00 2024-03-24T02:48:08.030 INFO:teuthology.orchestra.run.smithi017.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 407 kB/s | 27 kB 00:00 2024-03-24T02:48:08.106 INFO:teuthology.orchestra.run.smithi017.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 416 kB/s | 31 kB 00:00 2024-03-24T02:48:08.140 INFO:teuthology.orchestra.run.smithi019.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 38 kB/s | 21 kB 00:00 2024-03-24T02:48:08.173 INFO:teuthology.orchestra.run.smithi019.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 44 kB/s | 25 kB 00:00 2024-03-24T02:48:08.273 INFO:teuthology.orchestra.run.smithi017.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 4.4 MB/s | 746 kB 00:00 2024-03-24T02:48:08.356 INFO:teuthology.orchestra.run.smithi017.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.9 MB/s | 163 kB 00:00 2024-03-24T02:48:08.448 INFO:teuthology.orchestra.run.smithi017.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.4 MB/s | 127 kB 00:00 2024-03-24T02:48:08.548 INFO:teuthology.orchestra.run.smithi017.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 4.9 MB/s | 501 kB 00:00 2024-03-24T02:48:08.693 INFO:teuthology.orchestra.run.smithi188.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.2 MB/s | 890 kB 00:00 2024-03-24T02:48:08.952 INFO:teuthology.orchestra.run.smithi188.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 561 kB/s | 145 kB 00:00 2024-03-24T02:48:09.049 INFO:teuthology.orchestra.run.smithi019.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 270 kB/s | 245 kB 00:00 2024-03-24T02:48:09.074 INFO:teuthology.orchestra.run.smithi019.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 275 kB/s | 248 kB 00:00 2024-03-24T02:48:09.424 INFO:teuthology.orchestra.run.smithi017.stdout:(18/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 5.9 MB/s | 5.1 MB 00:00 2024-03-24T02:48:09.683 INFO:teuthology.orchestra.run.smithi019.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 57 kB/s | 35 kB 00:00 2024-03-24T02:48:09.857 INFO:teuthology.orchestra.run.smithi017.stdout:(19/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 16 MB/s | 48 MB 00:02 2024-03-24T02:48:09.869 INFO:teuthology.orchestra.run.smithi188.stdout:(4/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 2.5 MB/s | 2.3 MB 00:00 2024-03-24T02:48:09.882 INFO:teuthology.orchestra.run.smithi017.stdout:(20/174): python3-ceph-argparse-19.0.0-2145.g3b 98 kB/s | 45 kB 00:00 2024-03-24T02:48:09.907 INFO:teuthology.orchestra.run.smithi017.stdout:(21/174): python3-ceph-common-19.0.0-2145.g3be9 2.6 MB/s | 131 kB 00:00 2024-03-24T02:48:09.933 INFO:teuthology.orchestra.run.smithi017.stdout:(22/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 3.2 MB/s | 162 kB 00:00 2024-03-24T02:48:09.958 INFO:teuthology.orchestra.run.smithi017.stdout:(23/174): python3-rados-19.0.0-2145.g3be9a5c9.e 6.3 MB/s | 321 kB 00:00 2024-03-24T02:48:09.991 INFO:teuthology.orchestra.run.smithi019.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 230 kB/s | 216 kB 00:00 2024-03-24T02:48:09.992 INFO:teuthology.orchestra.run.smithi017.stdout:(24/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 5.0 MB/s | 298 kB 00:00 2024-03-24T02:48:10.017 INFO:teuthology.orchestra.run.smithi017.stdout:(25/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.7 MB/s | 99 kB 00:00 2024-03-24T02:48:10.042 INFO:teuthology.orchestra.run.smithi017.stdout:(26/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-24T02:48:10.126 INFO:teuthology.orchestra.run.smithi017.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-24T02:48:10.326 INFO:teuthology.orchestra.run.smithi017.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 170 kB/s | 34 kB 00:00 2024-03-24T02:48:10.344 INFO:teuthology.orchestra.run.smithi188.stdout:(5/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-03-24T02:48:10.377 INFO:teuthology.orchestra.run.smithi017.stdout:(29/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 8.6 MB/s | 3.1 MB 00:00 2024-03-24T02:48:10.377 INFO:teuthology.orchestra.run.smithi188.stdout:(6/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-03-24T02:48:10.391 INFO:teuthology.orchestra.run.smithi019.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 63 kB/s | 44 kB 00:00 2024-03-24T02:48:10.402 INFO:teuthology.orchestra.run.smithi017.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 1.8 MB/s | 139 kB 00:00 2024-03-24T02:48:10.533 INFO:teuthology.orchestra.run.smithi019.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 159 kB/s | 22 kB 00:00 2024-03-24T02:48:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 7.4 MB/s | 1.8 MB 00:00 2024-03-24T02:48:10.675 INFO:teuthology.orchestra.run.smithi019.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.9 MB/s | 863 kB 00:00 2024-03-24T02:48:10.736 INFO:teuthology.orchestra.run.smithi017.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 2.1 MB/s | 248 kB 00:00 2024-03-24T02:48:10.794 INFO:teuthology.orchestra.run.smithi017.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 845 kB/s | 49 kB 00:00 2024-03-24T02:48:10.842 INFO:teuthology.orchestra.run.smithi019.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 110 kB/s | 94 kB 00:00 2024-03-24T02:48:10.853 INFO:teuthology.orchestra.run.smithi017.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 282 kB/s | 16 kB 00:00 2024-03-24T02:48:10.978 INFO:teuthology.orchestra.run.smithi017.stdout:(35/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 2.1 MB/s | 264 kB 00:00 2024-03-24T02:48:11.226 INFO:teuthology.orchestra.run.smithi019.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 996 kB/s | 548 kB 00:00 2024-03-24T02:48:11.279 INFO:teuthology.orchestra.run.smithi017.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 8.4 MB/s | 7.4 MB 00:00 2024-03-24T02:48:11.304 INFO:teuthology.orchestra.run.smithi017.stdout:(37/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 2.2 MB/s | 748 kB 00:00 2024-03-24T02:48:11.309 INFO:teuthology.orchestra.run.smithi019.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-03-24T02:48:11.335 INFO:teuthology.orchestra.run.smithi019.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 629 kB/s | 309 kB 00:00 2024-03-24T02:48:11.338 INFO:teuthology.orchestra.run.smithi017.stdout:(38/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 312 kB/s | 18 kB 00:00 2024-03-24T02:48:11.410 INFO:teuthology.orchestra.run.smithi019.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 345 kB/s | 26 kB 00:00 2024-03-24T02:48:11.485 INFO:teuthology.orchestra.run.smithi019.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 650 kB/s | 49 kB 00:00 2024-03-24T02:48:11.571 INFO:teuthology.orchestra.run.smithi017.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-24T02:48:11.585 INFO:teuthology.orchestra.run.smithi019.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 674 kB/s | 67 kB 00:00 2024-03-24T02:48:11.629 INFO:teuthology.orchestra.run.smithi017.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-24T02:48:11.671 INFO:teuthology.orchestra.run.smithi017.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 920 kB 00:00 2024-03-24T02:48:11.686 INFO:teuthology.orchestra.run.smithi019.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-03-24T02:48:11.697 INFO:teuthology.orchestra.run.smithi017.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-03-24T02:48:11.714 INFO:teuthology.orchestra.run.smithi019.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-03-24T02:48:11.722 INFO:teuthology.orchestra.run.smithi017.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-03-24T02:48:11.777 INFO:teuthology.orchestra.run.smithi188.stdout:(7/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 3.3 MB/s | 4.7 MB 00:01 2024-03-24T02:48:11.806 INFO:teuthology.orchestra.run.smithi019.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.8 MB/s | 838 kB 00:00 2024-03-24T02:48:11.814 INFO:teuthology.orchestra.run.smithi017.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-24T02:48:11.839 INFO:teuthology.orchestra.run.smithi017.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 148 kB/s | 17 kB 00:00 2024-03-24T02:48:11.864 INFO:teuthology.orchestra.run.smithi017.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-03-24T02:48:11.890 INFO:teuthology.orchestra.run.smithi017.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-24T02:48:11.890 INFO:teuthology.orchestra.run.smithi019.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 522 kB/s | 43 kB 00:00 2024-03-24T02:48:11.907 INFO:teuthology.orchestra.run.smithi017.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-24T02:48:11.915 INFO:teuthology.orchestra.run.smithi019.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-03-24T02:48:11.940 INFO:teuthology.orchestra.run.smithi017.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-24T02:48:11.974 INFO:teuthology.orchestra.run.smithi019.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 386 kB/s | 32 kB 00:00 2024-03-24T02:48:11.982 INFO:teuthology.orchestra.run.smithi017.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-24T02:48:11.999 INFO:teuthology.orchestra.run.smithi019.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-03-24T02:48:11.999 INFO:teuthology.orchestra.run.smithi017.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 808 kB/s | 46 kB 00:00 2024-03-24T02:48:12.025 INFO:teuthology.orchestra.run.smithi017.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-24T02:48:12.050 INFO:teuthology.orchestra.run.smithi017.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-24T02:48:12.083 INFO:teuthology.orchestra.run.smithi019.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-03-24T02:48:12.083 INFO:teuthology.orchestra.run.smithi017.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-03-24T02:48:12.108 INFO:teuthology.orchestra.run.smithi019.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-03-24T02:48:12.109 INFO:teuthology.orchestra.run.smithi017.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 590 kB/s | 34 kB 00:00 2024-03-24T02:48:12.136 INFO:teuthology.orchestra.run.smithi188.stdout:(8/174): ceph-common-19.0.0-2145.g3be9a5c9.el9. 4.5 MB/s | 20 MB 00:04 2024-03-24T02:48:12.175 INFO:teuthology.orchestra.run.smithi019.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-03-24T02:48:12.186 INFO:teuthology.orchestra.run.smithi188.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 506 kB/s | 25 kB 00:00 2024-03-24T02:48:12.200 INFO:teuthology.orchestra.run.smithi019.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-03-24T02:48:12.218 INFO:teuthology.orchestra.run.smithi017.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-03-24T02:48:12.259 INFO:teuthology.orchestra.run.smithi017.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-03-24T02:48:12.302 INFO:teuthology.orchestra.run.smithi017.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 9.8 MB/s | 811 kB 00:00 2024-03-24T02:48:12.309 INFO:teuthology.orchestra.run.smithi019.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-03-24T02:48:12.327 INFO:teuthology.orchestra.run.smithi017.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2024-03-24T02:48:12.418 INFO:teuthology.orchestra.run.smithi019.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-03-24T02:48:12.486 INFO:teuthology.orchestra.run.smithi017.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2024-03-24T02:48:12.492 INFO:teuthology.orchestra.run.smithi019.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.0 MB/s | 2.0 MB 00:00 2024-03-24T02:48:12.519 INFO:teuthology.orchestra.run.smithi017.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-03-24T02:48:12.544 INFO:teuthology.orchestra.run.smithi017.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-24T02:48:12.551 INFO:teuthology.orchestra.run.smithi019.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-03-24T02:48:12.585 INFO:teuthology.orchestra.run.smithi019.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-03-24T02:48:12.635 INFO:teuthology.orchestra.run.smithi019.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-03-24T02:48:12.660 INFO:teuthology.orchestra.run.smithi019.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-03-24T02:48:12.695 INFO:teuthology.orchestra.run.smithi017.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-03-24T02:48:12.702 INFO:teuthology.orchestra.run.smithi019.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 295 kB/s | 20 kB 00:00 2024-03-24T02:48:12.712 INFO:teuthology.orchestra.run.smithi017.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 148 kB/s | 24 kB 00:00 2024-03-24T02:48:12.729 INFO:teuthology.orchestra.run.smithi017.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 541 kB/s | 18 kB 00:00 2024-03-24T02:48:12.754 INFO:teuthology.orchestra.run.smithi017.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-24T02:48:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-03-24T02:48:12.779 INFO:teuthology.orchestra.run.smithi017.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 634 kB/s | 32 kB 00:00 2024-03-24T02:48:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 504 kB/s | 46 kB 00:00 2024-03-24T02:48:12.870 INFO:teuthology.orchestra.run.smithi019.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-03-24T02:48:12.895 INFO:teuthology.orchestra.run.smithi019.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 574 kB/s | 58 kB 00:00 2024-03-24T02:48:12.946 INFO:teuthology.orchestra.run.smithi017.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-03-24T02:48:12.971 INFO:teuthology.orchestra.run.smithi017.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-03-24T02:48:12.979 INFO:teuthology.orchestra.run.smithi019.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-03-24T02:48:13.004 INFO:teuthology.orchestra.run.smithi019.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-03-24T02:48:13.005 INFO:teuthology.orchestra.run.smithi017.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 174 kB 00:00 2024-03-24T02:48:13.030 INFO:teuthology.orchestra.run.smithi017.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-24T02:48:13.038 INFO:teuthology.orchestra.run.smithi019.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-03-24T02:48:13.047 INFO:teuthology.orchestra.run.smithi017.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 985 kB/s | 41 kB 00:00 2024-03-24T02:48:13.072 INFO:teuthology.orchestra.run.smithi017.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-03-24T02:48:13.080 INFO:teuthology.orchestra.run.smithi019.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-03-24T02:48:13.130 INFO:teuthology.orchestra.run.smithi019.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-03-24T02:48:13.172 INFO:teuthology.orchestra.run.smithi019.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-03-24T02:48:13.206 INFO:teuthology.orchestra.run.smithi019.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-03-24T02:48:13.247 INFO:teuthology.orchestra.run.smithi019.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-03-24T02:48:13.281 INFO:teuthology.orchestra.run.smithi019.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-03-24T02:48:13.323 INFO:teuthology.orchestra.run.smithi019.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-24T02:48:13.348 INFO:teuthology.orchestra.run.smithi019.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-03-24T02:48:13.365 INFO:teuthology.orchestra.run.smithi017.stdout:(74/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 2.5 MB/s | 20 MB 00:08 2024-03-24T02:48:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 387 kB/s | 132 kB 00:00 2024-03-24T02:48:13.398 INFO:teuthology.orchestra.run.smithi019.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-03-24T02:48:13.415 INFO:teuthology.orchestra.run.smithi017.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 616 kB/s | 211 kB 00:00 2024-03-24T02:48:13.424 INFO:teuthology.orchestra.run.smithi019.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-03-24T02:48:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2024-03-24T02:48:13.449 INFO:teuthology.orchestra.run.smithi017.stdout:(78/174): python3-pytz-2021.1-5.el9.noarch.rpm 817 kB/s | 47 kB 00:00 2024-03-24T02:48:13.482 INFO:teuthology.orchestra.run.smithi019.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-03-24T02:48:13.574 INFO:teuthology.orchestra.run.smithi019.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-24T02:48:13.607 INFO:teuthology.orchestra.run.smithi017.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 222 kB/s | 39 kB 00:00 2024-03-24T02:48:13.632 INFO:teuthology.orchestra.run.smithi017.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 227 kB/s | 41 kB 00:00 2024-03-24T02:48:13.808 INFO:teuthology.orchestra.run.smithi017.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-03-24T02:48:13.833 INFO:teuthology.orchestra.run.smithi017.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 300 kB 00:00 2024-03-24T02:48:13.858 INFO:teuthology.orchestra.run.smithi017.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 253 kB/s | 56 kB 00:00 2024-03-24T02:48:13.883 INFO:teuthology.orchestra.run.smithi017.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 842 kB/s | 63 kB 00:00 2024-03-24T02:48:13.891 INFO:teuthology.orchestra.run.smithi019.stdout:(150/174): openssl-devel-3.0.7-27.el9.x86_64.rp 676 kB/s | 4.1 MB 00:06 2024-03-24T02:48:13.909 INFO:teuthology.orchestra.run.smithi017.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-24T02:48:13.933 INFO:teuthology.orchestra.run.smithi019.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-24T02:48:13.934 INFO:teuthology.orchestra.run.smithi017.stdout:(86/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 585 kB/s | 44 kB 00:00 2024-03-24T02:48:13.960 INFO:teuthology.orchestra.run.smithi017.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-24T02:48:14.001 INFO:teuthology.orchestra.run.smithi017.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-24T02:48:14.027 INFO:teuthology.orchestra.run.smithi017.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 1.9 MB/s | 124 kB 00:00 2024-03-24T02:48:14.052 INFO:teuthology.orchestra.run.smithi017.stdout:(90/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-24T02:48:14.077 INFO:teuthology.orchestra.run.smithi017.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-24T02:48:14.125 INFO:teuthology.orchestra.run.smithi019.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-03-24T02:48:14.201 INFO:teuthology.orchestra.run.smithi019.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 25 MB/s | 1.8 MB 00:00 2024-03-24T02:48:14.203 INFO:teuthology.orchestra.run.smithi017.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-24T02:48:14.228 INFO:teuthology.orchestra.run.smithi017.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-24T02:48:14.234 INFO:teuthology.orchestra.run.smithi019.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 42 kB 00:00 2024-03-24T02:48:14.253 INFO:teuthology.orchestra.run.smithi017.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-24T02:48:14.276 INFO:teuthology.orchestra.run.smithi019.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-24T02:48:14.286 INFO:teuthology.orchestra.run.smithi017.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-03-24T02:48:14.318 INFO:teuthology.orchestra.run.smithi019.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-24T02:48:14.337 INFO:teuthology.orchestra.run.smithi017.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 197 kB/s | 21 kB 00:00 2024-03-24T02:48:14.344 INFO:teuthology.orchestra.run.smithi188.stdout:(10/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 3.8 MB/s | 9.9 MB 00:02 2024-03-24T02:48:14.351 INFO:teuthology.orchestra.run.smithi019.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 1.1 MB/s | 36 kB 00:00 2024-03-24T02:48:14.362 INFO:teuthology.orchestra.run.smithi017.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-03-24T02:48:14.393 INFO:teuthology.orchestra.run.smithi019.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-03-24T02:48:14.435 INFO:teuthology.orchestra.run.smithi019.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-03-24T02:48:14.444 INFO:teuthology.orchestra.run.smithi188.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 2.2 MB/s | 220 kB 00:00 2024-03-24T02:48:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 552 kB/s | 23 kB 00:00 2024-03-24T02:48:14.504 INFO:teuthology.orchestra.run.smithi017.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-03-24T02:48:14.528 INFO:teuthology.orchestra.run.smithi188.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 325 kB/s | 27 kB 00:00 2024-03-24T02:48:14.529 INFO:teuthology.orchestra.run.smithi017.stdout:(99/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-03-24T02:48:14.536 INFO:teuthology.orchestra.run.smithi019.stdout:(161/174): librados2-19.0.0-2145.g3be9a5c9.el9. 3.2 MB/s | 3.4 MB 00:01 2024-03-24T02:48:14.561 INFO:teuthology.orchestra.run.smithi019.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 108 kB 00:00 2024-03-24T02:48:14.571 INFO:teuthology.orchestra.run.smithi017.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 837 kB/s | 35 kB 00:00 2024-03-24T02:48:14.596 INFO:teuthology.orchestra.run.smithi017.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.3 MB/s | 216 kB 00:00 2024-03-24T02:48:14.603 INFO:teuthology.orchestra.run.smithi019.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 136 kB 00:00 2024-03-24T02:48:14.614 INFO:teuthology.orchestra.run.smithi017.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-24T02:48:14.637 INFO:teuthology.orchestra.run.smithi019.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-24T02:48:14.647 INFO:teuthology.orchestra.run.smithi017.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-24T02:48:14.662 INFO:teuthology.orchestra.run.smithi019.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-24T02:48:14.687 INFO:teuthology.orchestra.run.smithi019.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 574 kB/s | 29 kB 00:00 2024-03-24T02:48:14.730 INFO:teuthology.orchestra.run.smithi019.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 18 MB/s | 1.2 MB 00:00 2024-03-24T02:48:14.788 INFO:teuthology.orchestra.run.smithi019.stdout:(168/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 2.6 MB/s | 3.1 MB 00:01 2024-03-24T02:48:14.850 INFO:teuthology.orchestra.run.smithi019.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 20 MB/s | 2.3 MB 00:00 2024-03-24T02:48:14.889 INFO:teuthology.orchestra.run.smithi019.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-03-24T02:48:14.914 INFO:teuthology.orchestra.run.smithi019.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 467 kB/s | 31 kB 00:00 2024-03-24T02:48:14.948 INFO:teuthology.orchestra.run.smithi019.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-03-24T02:48:14.973 INFO:teuthology.orchestra.run.smithi019.stdout:(173/174): util-linux-core-2.37.4-18.el9.x86_64 2.5 MB/s | 465 kB 00:00 2024-03-24T02:48:15.004 INFO:teuthology.orchestra.run.smithi188.stdout:(13/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 17 MB/s | 48 MB 00:02 2024-03-24T02:48:15.020 INFO:teuthology.orchestra.run.smithi188.stdout:(14/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 64 kB/s | 31 kB 00:00 2024-03-24T02:48:15.104 INFO:teuthology.orchestra.run.smithi188.stdout:(15/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 7.3 MB/s | 746 kB 00:00 2024-03-24T02:48:15.106 INFO:teuthology.orchestra.run.smithi017.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 5.0 MB/s | 4.1 MB 00:00 2024-03-24T02:48:15.129 INFO:teuthology.orchestra.run.smithi188.stdout:(16/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.5 MB/s | 163 kB 00:00 2024-03-24T02:48:15.171 INFO:teuthology.orchestra.run.smithi188.stdout:(17/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.9 MB/s | 127 kB 00:00 2024-03-24T02:48:15.189 INFO:teuthology.orchestra.run.smithi017.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-03-24T02:48:15.288 INFO:teuthology.orchestra.run.smithi188.stdout:(18/174): libradosstriper1-19.0.0-2145.g3be9a5c 3.1 MB/s | 501 kB 00:00 2024-03-24T02:48:15.380 INFO:teuthology.orchestra.run.smithi188.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 493 kB/s | 45 kB 00:00 2024-03-24T02:48:15.497 INFO:teuthology.orchestra.run.smithi188.stdout:(20/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 16 MB/s | 5.1 MB 00:00 2024-03-24T02:48:15.522 INFO:teuthology.orchestra.run.smithi188.stdout:(21/174): python3-ceph-common-19.0.0-2145.g3be9 922 kB/s | 131 kB 00:00 2024-03-24T02:48:15.565 INFO:teuthology.orchestra.run.smithi188.stdout:(22/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 2.4 MB/s | 162 kB 00:00 2024-03-24T02:48:15.623 INFO:teuthology.orchestra.run.smithi019.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 105 kB/s | 74 kB 00:00 2024-03-24T02:48:15.631 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-24T02:48:15.631 INFO:teuthology.orchestra.run.smithi019.stdout:Total 14 MB/s | 224 MB 00:16 2024-03-24T02:48:15.648 INFO:teuthology.orchestra.run.smithi188.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 3.5 MB/s | 298 kB 00:00 2024-03-24T02:48:15.682 INFO:teuthology.orchestra.run.smithi188.stdout:(24/174): python3-rados-19.0.0-2145.g3be9a5c9.e 2.0 MB/s | 321 kB 00:00 2024-03-24T02:48:15.715 INFO:teuthology.orchestra.run.smithi188.stdout:(25/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.5 MB/s | 99 kB 00:00 2024-03-24T02:48:15.741 INFO:teuthology.orchestra.run.smithi188.stdout:(26/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-24T02:48:15.824 INFO:teuthology.orchestra.run.smithi188.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-24T02:48:15.924 INFO:teuthology.orchestra.run.smithi188.stdout:(28/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 15 MB/s | 3.1 MB 00:00 2024-03-24T02:48:15.950 INFO:teuthology.orchestra.run.smithi188.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2145.g 271 kB/s | 34 kB 00:00 2024-03-24T02:48:16.017 INFO:teuthology.orchestra.run.smithi188.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 2.0 MB/s | 139 kB 00:00 2024-03-24T02:48:16.106 INFO:teuthology.orchestra.run.smithi017.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 347 kB/s | 309 kB 00:00 2024-03-24T02:48:16.109 INFO:teuthology.orchestra.run.smithi188.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 11 MB/s | 1.8 MB 00:00 2024-03-24T02:48:16.123 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-24T02:48:16.175 INFO:teuthology.orchestra.run.smithi188.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 3.7 MB/s | 248 kB 00:00 2024-03-24T02:48:16.258 INFO:teuthology.orchestra.run.smithi188.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 597 kB/s | 49 kB 00:00 2024-03-24T02:48:16.276 INFO:teuthology.orchestra.run.smithi017.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 115 kB/s | 19 kB 00:00 2024-03-24T02:48:16.307 INFO:teuthology.orchestra.run.smithi017.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 457 kB/s | 548 kB 00:01 2024-03-24T02:48:16.318 INFO:teuthology.orchestra.run.smithi188.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 278 kB/s | 16 kB 00:00 2024-03-24T02:48:16.416 INFO:teuthology.orchestra.run.smithi017.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-03-24T02:48:16.422 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-24T02:48:16.422 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-24T02:48:16.457 INFO:teuthology.orchestra.run.smithi017.stdout:(110/174): protobuf-compiler-3.14.0-13.el9.x86_ 477 kB/s | 863 kB 00:01 2024-03-24T02:48:16.487 INFO:teuthology.orchestra.run.smithi188.stdout:(35/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 2.8 MB/s | 17 MB 00:06 2024-03-24T02:48:16.512 INFO:teuthology.orchestra.run.smithi188.stdout:(36/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 1.3 MB/s | 264 kB 00:00 2024-03-24T02:48:16.524 INFO:teuthology.orchestra.run.smithi017.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 452 kB/s | 49 kB 00:00 2024-03-24T02:48:16.571 INFO:teuthology.orchestra.run.smithi188.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 313 kB/s | 18 kB 00:00 2024-03-24T02:48:16.763 INFO:teuthology.orchestra.run.smithi188.stdout:(38/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 2.7 MB/s | 748 kB 00:00 2024-03-24T02:48:16.799 INFO:teuthology.orchestra.run.smithi017.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 550 kB/s | 151 kB 00:00 2024-03-24T02:48:17.016 INFO:teuthology.orchestra.run.smithi017.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 121 kB/s | 67 kB 00:00 2024-03-24T02:48:17.030 INFO:teuthology.orchestra.run.smithi188.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.4 MB/s | 662 kB 00:00 2024-03-24T02:48:17.164 INFO:teuthology.orchestra.run.smithi188.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 920 kB 00:00 2024-03-24T02:48:17.189 INFO:teuthology.orchestra.run.smithi188.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 669 kB/s | 106 kB 00:00 2024-03-24T02:48:17.231 INFO:teuthology.orchestra.run.smithi188.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-03-24T02:48:17.256 INFO:teuthology.orchestra.run.smithi188.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 475 kB/s | 32 kB 00:00 2024-03-24T02:48:17.440 INFO:teuthology.orchestra.run.smithi188.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 94 kB/s | 17 kB 00:00 2024-03-24T02:48:17.532 INFO:teuthology.orchestra.run.smithi188.stdout:(45/174): ceph-mgr-diskprediction-local-19.0.0- 4.9 MB/s | 7.4 MB 00:01 2024-03-24T02:48:17.582 INFO:teuthology.orchestra.run.smithi188.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.5 MB/s | 3.0 MB 00:00 2024-03-24T02:48:17.607 INFO:teuthology.orchestra.run.smithi188.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 60 kB/s | 10 kB 00:00 2024-03-24T02:48:17.632 INFO:teuthology.orchestra.run.smithi188.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 159 kB 00:00 2024-03-24T02:48:17.658 INFO:teuthology.orchestra.run.smithi188.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 886 kB/s | 44 kB 00:00 2024-03-24T02:48:17.700 INFO:teuthology.orchestra.run.smithi188.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 661 kB 00:00 2024-03-24T02:48:17.742 INFO:teuthology.orchestra.run.smithi188.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 578 kB 00:00 2024-03-24T02:48:17.767 INFO:teuthology.orchestra.run.smithi188.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 428 kB/s | 46 kB 00:00 2024-03-24T02:48:17.791 INFO:teuthology.orchestra.run.smithi017.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 707 kB/s | 548 kB 00:00 2024-03-24T02:48:17.792 INFO:teuthology.orchestra.run.smithi188.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-03-24T02:48:17.818 INFO:teuthology.orchestra.run.smithi188.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-24T02:48:17.843 INFO:teuthology.orchestra.run.smithi188.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 688 kB/s | 34 kB 00:00 2024-03-24T02:48:17.868 INFO:teuthology.orchestra.run.smithi188.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-24T02:48:17.917 INFO:teuthology.orchestra.run.smithi017.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 750 kB/s | 838 kB 00:01 2024-03-24T02:48:17.942 INFO:teuthology.orchestra.run.smithi017.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 289 kB/s | 43 kB 00:00 2024-03-24T02:48:18.025 INFO:teuthology.orchestra.run.smithi017.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-03-24T02:48:18.059 INFO:teuthology.orchestra.run.smithi017.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-03-24T02:48:18.241 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-24T02:48:18.241 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-24T02:48:18.243 INFO:teuthology.orchestra.run.smithi188.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-03-24T02:48:18.277 INFO:teuthology.orchestra.run.smithi188.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 1.9 MB/s | 811 kB 00:00 2024-03-24T02:48:18.293 INFO:teuthology.orchestra.run.smithi017.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 643 kB/s | 172 kB 00:00 2024-03-24T02:48:18.319 INFO:teuthology.orchestra.run.smithi188.stdout:(59/174): postgresql-13.11-1.el9.x86_64.rpm 3.2 MB/s | 1.5 MB 00:00 2024-03-24T02:48:18.344 INFO:teuthology.orchestra.run.smithi188.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2024-03-24T02:48:18.536 INFO:teuthology.orchestra.run.smithi188.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 22 MB/s | 5.7 MB 00:00 2024-03-24T02:48:18.561 INFO:teuthology.orchestra.run.smithi188.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 364 kB/s | 78 kB 00:00 2024-03-24T02:48:18.577 INFO:teuthology.orchestra.run.smithi017.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 693 kB/s | 358 kB 00:00 2024-03-24T02:48:18.603 INFO:teuthology.orchestra.run.smithi188.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-03-24T02:48:18.685 INFO:teuthology.orchestra.run.smithi017.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-03-24T02:48:18.727 INFO:teuthology.orchestra.run.smithi017.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 698 kB/s | 303 kB 00:00 2024-03-24T02:48:18.729 INFO:teuthology.orchestra.run.smithi188.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 5.8 MB 00:00 2024-03-24T02:48:18.745 INFO:teuthology.orchestra.run.smithi188.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 136 kB/s | 24 kB 00:00 2024-03-24T02:48:18.770 INFO:teuthology.orchestra.run.smithi188.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 110 kB/s | 18 kB 00:00 2024-03-24T02:48:18.796 INFO:teuthology.orchestra.run.smithi188.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-03-24T02:48:18.821 INFO:teuthology.orchestra.run.smithi188.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-24T02:48:18.930 INFO:teuthology.orchestra.run.smithi188.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-24T02:48:18.955 INFO:teuthology.orchestra.run.smithi188.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-24T02:48:18.961 INFO:teuthology.orchestra.run.smithi017.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 737 kB/s | 203 kB 00:00 2024-03-24T02:48:18.980 INFO:teuthology.orchestra.run.smithi188.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2024-03-24T02:48:19.006 INFO:teuthology.orchestra.run.smithi188.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 920 kB/s | 69 kB 00:00 2024-03-24T02:48:19.031 INFO:teuthology.orchestra.run.smithi188.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 827 kB/s | 41 kB 00:00 2024-03-24T02:48:19.064 INFO:teuthology.orchestra.run.smithi188.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-24T02:48:19.090 INFO:teuthology.orchestra.run.smithi188.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-03-24T02:48:19.115 INFO:teuthology.orchestra.run.smithi188.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-24T02:48:19.140 INFO:teuthology.orchestra.run.smithi188.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 631 kB/s | 47 kB 00:00 2024-03-24T02:48:19.165 INFO:teuthology.orchestra.run.smithi188.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 573 kB/s | 43 kB 00:00 2024-03-24T02:48:19.194 INFO:teuthology.orchestra.run.smithi188.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 719 kB/s | 39 kB 00:00 2024-03-24T02:48:19.228 INFO:teuthology.orchestra.run.smithi017.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 541 kB/s | 144 kB 00:00 2024-03-24T02:48:19.332 INFO:teuthology.orchestra.run.smithi188.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 246 kB/s | 41 kB 00:00 2024-03-24T02:48:19.345 INFO:teuthology.orchestra.run.smithi017.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-03-24T02:48:19.366 INFO:teuthology.orchestra.run.smithi188.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 300 kB 00:00 2024-03-24T02:48:19.453 INFO:teuthology.orchestra.run.smithi017.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-03-24T02:48:19.525 INFO:teuthology.orchestra.run.smithi188.stdout:(82/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-03-24T02:48:19.550 INFO:teuthology.orchestra.run.smithi188.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 262 kB/s | 56 kB 00:00 2024-03-24T02:48:19.562 INFO:teuthology.orchestra.run.smithi017.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-03-24T02:48:19.576 INFO:teuthology.orchestra.run.smithi188.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 303 kB/s | 63 kB 00:00 2024-03-24T02:48:19.601 INFO:teuthology.orchestra.run.smithi188.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 877 kB/s | 44 kB 00:00 2024-03-24T02:48:19.626 INFO:teuthology.orchestra.run.smithi188.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-24T02:48:19.651 INFO:teuthology.orchestra.run.smithi188.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-24T02:48:19.670 INFO:teuthology.orchestra.run.smithi017.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-03-24T02:48:19.677 INFO:teuthology.orchestra.run.smithi188.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-03-24T02:48:19.712 INFO:teuthology.orchestra.run.smithi188.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-24T02:48:19.738 INFO:teuthology.orchestra.run.smithi188.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-03-24T02:48:19.762 INFO:teuthology.orchestra.run.smithi017.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 215 kB/s | 20 kB 00:00 2024-03-24T02:48:19.763 INFO:teuthology.orchestra.run.smithi188.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-24T02:48:20.113 INFO:teuthology.orchestra.run.smithi188.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 558 kB/s | 195 kB 00:00 2024-03-24T02:48:20.180 INFO:teuthology.orchestra.run.smithi188.stdout:(93/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-24T02:48:20.205 INFO:teuthology.orchestra.run.smithi188.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-03-24T02:48:20.231 INFO:teuthology.orchestra.run.smithi188.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 380 kB/s | 187 kB 00:00 2024-03-24T02:48:20.464 INFO:teuthology.orchestra.run.smithi188.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 76 kB/s | 21 kB 00:00 2024-03-24T02:48:20.490 INFO:teuthology.orchestra.run.smithi188.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 90 kB/s | 25 kB 00:00 2024-03-24T02:48:20.965 INFO:teuthology.orchestra.run.smithi188.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 490 kB/s | 245 kB 00:00 2024-03-24T02:48:21.071 INFO:teuthology.orchestra.run.smithi017.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 799 kB/s | 1.0 MB 00:01 2024-03-24T02:48:21.171 INFO:teuthology.orchestra.run.smithi017.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-03-24T02:48:21.232 INFO:teuthology.orchestra.run.smithi188.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 334 kB/s | 248 kB 00:00 2024-03-24T02:48:21.280 INFO:teuthology.orchestra.run.smithi017.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 728 kB/s | 79 kB 00:00 2024-03-24T02:48:21.389 INFO:teuthology.orchestra.run.smithi017.stdout:(133/174): python3-grpcio-1.46.7-10.el9.x86_64. 784 kB/s | 2.0 MB 00:02 2024-03-24T02:48:21.406 INFO:teuthology.orchestra.run.smithi017.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 461 kB/s | 58 kB 00:00 2024-03-24T02:48:21.498 INFO:teuthology.orchestra.run.smithi188.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 131 kB/s | 35 kB 00:00 2024-03-24T02:48:21.514 INFO:teuthology.orchestra.run.smithi017.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-03-24T02:48:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 376 kB/s | 216 kB 00:00 2024-03-24T02:48:21.706 INFO:teuthology.orchestra.run.smithi017.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 469 kB/s | 90 kB 00:00 2024-03-24T02:48:21.806 INFO:teuthology.orchestra.run.smithi017.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-03-24T02:48:21.832 INFO:teuthology.orchestra.run.smithi017.stdout:(138/174): python3-pecan-1.4.2-3.el9.noarch.rpm 615 kB/s | 272 kB 00:00 2024-03-24T02:48:21.874 INFO:teuthology.orchestra.run.smithi188.stdout:(102/174): openssl-devel-3.0.7-27.el9.x86_64.rp 2.5 MB/s | 4.1 MB 00:01 2024-03-24T02:48:21.899 INFO:teuthology.orchestra.run.smithi188.stdout:(103/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 111 kB/s | 44 kB 00:00 2024-03-24T02:48:21.915 INFO:teuthology.orchestra.run.smithi017.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2024-03-24T02:48:21.958 INFO:teuthology.orchestra.run.smithi188.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-03-24T02:48:21.983 INFO:teuthology.orchestra.run.smithi188.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 212 kB/s | 94 kB 00:00 2024-03-24T02:48:22.049 INFO:teuthology.orchestra.run.smithi017.stdout:(140/174): libarrow-9.0.0-10.el9.x86_64.rpm 784 kB/s | 4.4 MB 00:05 2024-03-24T02:48:22.050 INFO:teuthology.orchestra.run.smithi188.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-03-24T02:48:22.066 INFO:teuthology.orchestra.run.smithi017.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 393 kB/s | 59 kB 00:00 2024-03-24T02:48:22.149 INFO:teuthology.orchestra.run.smithi017.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 594 kB/s | 188 kB 00:00 2024-03-24T02:48:22.166 INFO:teuthology.orchestra.run.smithi017.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-03-24T02:48:22.183 INFO:teuthology.orchestra.run.smithi017.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 435 kB/s | 51 kB 00:00 2024-03-24T02:48:22.200 INFO:teuthology.orchestra.run.smithi188.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 130 kB/s | 19 kB 00:00 2024-03-24T02:48:22.284 INFO:teuthology.orchestra.run.smithi188.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-03-24T02:48:22.326 INFO:teuthology.orchestra.run.smithi188.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-24T02:48:22.350 INFO:teuthology.orchestra.run.smithi017.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 488 kB/s | 90 kB 00:00 2024-03-24T02:48:22.351 INFO:teuthology.orchestra.run.smithi188.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-03-24T02:48:22.393 INFO:teuthology.orchestra.run.smithi188.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 727 kB/s | 49 kB 00:00 2024-03-24T02:48:22.418 INFO:teuthology.orchestra.run.smithi188.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-24T02:48:22.450 INFO:teuthology.orchestra.run.smithi017.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-03-24T02:48:22.460 INFO:teuthology.orchestra.run.smithi188.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-03-24T02:48:22.492 INFO:teuthology.orchestra.run.smithi017.stdout:(147/174): python3-webob-1.8.7-6.el9.noarch.rpm 670 kB/s | 230 kB 00:00 2024-03-24T02:48:22.536 INFO:teuthology.orchestra.run.smithi188.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.0 MB/s | 838 kB 00:00 2024-03-24T02:48:22.594 INFO:teuthology.orchestra.run.smithi188.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-03-24T02:48:22.628 INFO:teuthology.orchestra.run.smithi188.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 471 kB/s | 43 kB 00:00 2024-03-24T02:48:22.695 INFO:teuthology.orchestra.run.smithi188.stdout:(117/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 4.4 MB 00:00 2024-03-24T02:48:22.721 INFO:teuthology.orchestra.run.smithi188.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-03-24T02:48:22.726 INFO:teuthology.orchestra.run.smithi017.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 696 kB/s | 192 kB 00:00 2024-03-24T02:48:22.746 INFO:teuthology.orchestra.run.smithi188.stdout:(119/174): python3-cachetools-4.2.4-1.el9.noarc 213 kB/s | 32 kB 00:00 2024-03-24T02:48:22.751 INFO:teuthology.orchestra.run.smithi017.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 752 kB/s | 427 kB 00:00 2024-03-24T02:48:22.771 INFO:teuthology.orchestra.run.smithi188.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-03-24T02:48:22.805 INFO:teuthology.orchestra.run.smithi188.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-24T02:48:22.830 INFO:teuthology.orchestra.run.smithi188.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-03-24T02:48:22.855 INFO:teuthology.orchestra.run.smithi188.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 420 kB/s | 35 kB 00:00 2024-03-24T02:48:22.880 INFO:teuthology.orchestra.run.smithi188.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-24T02:48:22.922 INFO:teuthology.orchestra.run.smithi188.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-03-24T02:48:22.973 INFO:teuthology.orchestra.run.smithi188.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-03-24T02:48:22.998 INFO:teuthology.orchestra.run.smithi188.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-03-24T02:48:23.023 INFO:teuthology.orchestra.run.smithi188.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-03-24T02:48:23.049 INFO:teuthology.orchestra.run.smithi188.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-03-24T02:48:23.074 INFO:teuthology.orchestra.run.smithi188.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-03-24T02:48:23.099 INFO:teuthology.orchestra.run.smithi188.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-03-24T02:48:23.133 INFO:teuthology.orchestra.run.smithi188.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-03-24T02:48:23.150 INFO:teuthology.orchestra.run.smithi188.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 612 kB/s | 46 kB 00:00 2024-03-24T02:48:23.175 INFO:teuthology.orchestra.run.smithi188.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-03-24T02:48:23.202 INFO:teuthology.orchestra.run.smithi188.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 860 kB/s | 58 kB 00:00 2024-03-24T02:48:23.234 INFO:teuthology.orchestra.run.smithi188.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-24T02:48:23.251 INFO:teuthology.orchestra.run.smithi188.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-03-24T02:48:23.285 INFO:teuthology.orchestra.run.smithi188.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-03-24T02:48:23.310 INFO:teuthology.orchestra.run.smithi188.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-03-24T02:48:23.335 INFO:teuthology.orchestra.run.smithi188.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-03-24T02:48:23.360 INFO:teuthology.orchestra.run.smithi188.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-24T02:48:23.377 INFO:teuthology.orchestra.run.smithi017.stdout:(150/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 5.0 MB/s | 3.1 MB 00:00 2024-03-24T02:48:23.386 INFO:teuthology.orchestra.run.smithi188.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-03-24T02:48:23.411 INFO:teuthology.orchestra.run.smithi188.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-03-24T02:48:23.419 INFO:teuthology.orchestra.run.smithi017.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-24T02:48:23.445 INFO:teuthology.orchestra.run.smithi188.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 606 kB/s | 51 kB 00:00 2024-03-24T02:48:23.470 INFO:teuthology.orchestra.run.smithi188.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-24T02:48:23.495 INFO:teuthology.orchestra.run.smithi188.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-24T02:48:23.521 INFO:teuthology.orchestra.run.smithi188.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-03-24T02:48:23.546 INFO:teuthology.orchestra.run.smithi188.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-03-24T02:48:23.552 INFO:teuthology.orchestra.run.smithi017.stdout:(152/174): librados2-19.0.0-2145.g3be9a5c9.el9. 4.1 MB/s | 3.4 MB 00:00 2024-03-24T02:48:23.571 INFO:teuthology.orchestra.run.smithi188.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-24T02:48:23.630 INFO:teuthology.orchestra.run.smithi188.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-03-24T02:48:23.644 INFO:teuthology.orchestra.run.smithi017.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 10 MB/s | 2.3 MB 00:00 2024-03-24T02:48:23.672 INFO:teuthology.orchestra.run.smithi188.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-24T02:48:23.686 INFO:teuthology.orchestra.run.smithi017.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-24T02:48:23.728 INFO:teuthology.orchestra.run.smithi017.stdout:(155/174): NetworkManager-libnm-1.47.2-1.el9.x8 10 MB/s | 1.8 MB 00:00 2024-03-24T02:48:23.783 INFO:teuthology.orchestra.run.smithi017.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.5 MB/s | 248 kB 00:00 2024-03-24T02:48:23.812 INFO:teuthology.orchestra.run.smithi017.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 854 kB/s | 71 kB 00:00 2024-03-24T02:48:23.837 INFO:teuthology.orchestra.run.smithi017.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 676 kB/s | 36 kB 00:00 2024-03-24T02:48:23.862 INFO:teuthology.orchestra.run.smithi017.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 192 kB 00:00 2024-03-24T02:48:23.899 INFO:teuthology.orchestra.run.smithi017.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.0 MB/s | 256 kB 00:00 2024-03-24T02:48:23.929 INFO:teuthology.orchestra.run.smithi017.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 344 kB/s | 23 kB 00:00 2024-03-24T02:48:23.963 INFO:teuthology.orchestra.run.smithi017.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.7 MB/s | 108 kB 00:00 2024-03-24T02:48:23.988 INFO:teuthology.orchestra.run.smithi017.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-03-24T02:48:24.013 INFO:teuthology.orchestra.run.smithi017.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-24T02:48:24.039 INFO:teuthology.orchestra.run.smithi017.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-24T02:48:24.056 INFO:teuthology.orchestra.run.smithi017.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 679 kB/s | 29 kB 00:00 2024-03-24T02:48:24.106 INFO:teuthology.orchestra.run.smithi017.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 18 MB/s | 1.2 MB 00:00 2024-03-24T02:48:24.156 INFO:teuthology.orchestra.run.smithi017.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 22 MB/s | 2.2 MB 00:00 2024-03-24T02:48:24.164 INFO:teuthology.orchestra.run.smithi188.stdout:(152/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 5.3 MB/s | 3.1 MB 00:00 2024-03-24T02:48:24.214 INFO:teuthology.orchestra.run.smithi188.stdout:(153/174): librados2-19.0.0-2145.g3be9a5c9.el9. 5.1 MB/s | 3.4 MB 00:00 2024-03-24T02:48:24.223 INFO:teuthology.orchestra.run.smithi017.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 20 MB/s | 2.3 MB 00:00 2024-03-24T02:48:24.249 INFO:teuthology.orchestra.run.smithi017.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 4.9 MB/s | 465 kB 00:00 2024-03-24T02:48:24.265 INFO:teuthology.orchestra.run.smithi017.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 745 kB/s | 31 kB 00:00 2024-03-24T02:48:24.316 INFO:teuthology.orchestra.run.smithi017.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-03-24T02:48:24.349 INFO:teuthology.orchestra.run.smithi017.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 891 kB/s | 74 kB 00:00 2024-03-24T02:48:24.456 INFO:teuthology.orchestra.run.smithi188.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 174 kB/s | 42 kB 00:00 2024-03-24T02:48:24.466 INFO:teuthology.orchestra.run.smithi017.stdout:(174/174): thrift-0.15.0-2.el9.x86_64.rpm 821 kB/s | 1.6 MB 00:01 2024-03-24T02:48:24.474 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-03-24T02:48:24.474 INFO:teuthology.orchestra.run.smithi017.stdout:Total 11 MB/s | 224 MB 00:19 2024-03-24T02:48:24.928 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-03-24T02:48:25.140 INFO:teuthology.orchestra.run.smithi188.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 363 kB/s | 248 kB 00:00 2024-03-24T02:48:25.223 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-03-24T02:48:25.224 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-03-24T02:48:25.270 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-24T02:48:25.340 INFO:teuthology.orchestra.run.smithi188.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 357 kB/s | 71 kB 00:00 2024-03-24T02:48:25.377 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-24T02:48:25.403 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-24T02:48:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-24T02:48:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-24T02:48:25.456 INFO:teuthology.orchestra.run.smithi188.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 313 kB/s | 36 kB 00:00 2024-03-24T02:48:25.484 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-24T02:48:25.538 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-24T02:48:25.571 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-24T02:48:25.596 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-24T02:48:25.639 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-24T02:48:25.865 INFO:teuthology.orchestra.run.smithi188.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 471 kB/s | 192 kB 00:00 2024-03-24T02:48:25.976 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-24T02:48:26.038 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-24T02:48:26.146 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-24T02:48:26.173 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-24T02:48:26.265 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-24T02:48:26.294 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-24T02:48:26.332 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-24T02:48:26.466 INFO:teuthology.orchestra.run.smithi188.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 426 kB/s | 256 kB 00:00 2024-03-24T02:48:26.510 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-24T02:48:26.558 INFO:teuthology.orchestra.run.smithi188.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 252 kB/s | 23 kB 00:00 2024-03-24T02:48:26.559 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-24T02:48:26.586 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-24T02:48:26.675 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-24T02:48:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-24T02:48:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-24T02:48:26.807 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-24T02:48:26.830 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-24T02:48:26.841 INFO:teuthology.orchestra.run.smithi188.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 383 kB/s | 108 kB 00:00 2024-03-24T02:48:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-24T02:48:27.040 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-24T02:48:27.070 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-24T02:48:27.118 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-24T02:48:27.227 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-24T02:48:27.258 INFO:teuthology.orchestra.run.smithi188.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 371 kB/s | 155 kB 00:00 2024-03-24T02:48:27.337 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-24T02:48:27.347 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-03-24T02:48:27.347 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-03-24T02:48:27.358 INFO:teuthology.orchestra.run.smithi188.stdout:(163/174): NetworkManager-libnm-1.47.2-1.el9.x8 587 kB/s | 1.8 MB 00:03 2024-03-24T02:48:27.397 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-24T02:48:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-24T02:48:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-24T02:48:27.525 INFO:teuthology.orchestra.run.smithi188.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 383 kB/s | 64 kB 00:00 2024-03-24T02:48:27.667 INFO:teuthology.orchestra.run.smithi188.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 332 kB/s | 136 kB 00:00 2024-03-24T02:48:27.741 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-24T02:48:27.849 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-24T02:48:27.945 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-24T02:48:27.986 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-24T02:48:28.013 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-24T02:48:28.034 INFO:teuthology.orchestra.run.smithi188.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 56 kB/s | 29 kB 00:00 2024-03-24T02:48:28.049 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-24T02:48:28.127 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-24T02:48:28.178 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-24T02:48:28.216 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-24T02:48:28.292 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-24T02:48:28.331 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-24T02:48:28.375 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-24T02:48:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-24T02:48:28.435 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-24T02:48:28.508 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-24T02:48:28.540 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-24T02:48:28.584 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-24T02:48:28.612 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-24T02:48:28.685 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-24T02:48:28.727 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-24T02:48:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-24T02:48:28.953 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-24T02:48:29.006 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-24T02:48:29.566 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-24T02:48:29.645 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-24T02:48:29.736 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-24T02:48:29.794 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-24T02:48:29.803 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-24T02:48:29.834 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-24T02:48:29.884 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-24T02:48:29.922 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-24T02:48:29.951 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-24T02:48:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-24T02:48:30.365 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-24T02:48:30.437 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-24T02:48:30.473 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-24T02:48:30.614 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-24T02:48:30.659 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-24T02:48:30.715 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-24T02:48:30.799 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-24T02:48:30.928 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-24T02:48:30.942 INFO:teuthology.orchestra.run.smithi188.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 378 kB/s | 1.2 MB 00:03 2024-03-24T02:48:30.984 INFO:teuthology.orchestra.run.smithi188.stdout:(168/174): NetworkManager-1.47.2-1.el9.x86_64.r 323 kB/s | 2.3 MB 00:07 2024-03-24T02:48:31.064 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-24T02:48:31.112 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-24T02:48:31.146 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-24T02:48:31.191 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-24T02:48:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-24T02:48:31.375 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-24T02:48:31.429 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-24T02:48:31.459 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-24T02:48:31.501 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-24T02:48:31.608 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-24T02:48:31.641 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-24T02:48:31.682 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-24T02:48:31.707 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-24T02:48:31.732 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-24T02:48:31.742 INFO:teuthology.orchestra.run.smithi188.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 595 kB/s | 2.2 MB 00:03 2024-03-24T02:48:31.807 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-24T02:48:31.843 INFO:teuthology.orchestra.run.smithi188.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 311 kB/s | 31 kB 00:00 2024-03-24T02:48:31.938 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-24T02:48:32.054 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-24T02:48:32.099 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-24T02:48:32.157 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-24T02:48:32.168 INFO:teuthology.orchestra.run.smithi188.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 393 kB/s | 465 kB 00:01 2024-03-24T02:48:32.221 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-24T02:48:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-24T02:48:32.470 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-24T02:48:32.518 INFO:teuthology.orchestra.run.smithi188.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 212 kB/s | 74 kB 00:00 2024-03-24T02:48:32.615 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-24T02:48:32.660 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-24T02:48:32.691 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-24T02:48:32.730 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-24T02:48:32.758 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-24T02:48:32.793 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-24T02:48:32.832 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-24T02:48:32.863 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-24T02:48:32.893 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-24T02:48:32.972 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-24T02:48:33.012 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-24T02:48:33.046 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-24T02:48:33.087 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-24T02:48:33.113 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-24T02:48:33.208 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-24T02:48:33.296 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-24T02:48:33.344 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-24T02:48:33.376 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-24T02:48:33.423 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-24T02:48:33.524 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-24T02:48:33.578 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-24T02:48:34.349 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-24T02:48:34.391 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-24T02:48:34.512 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-24T02:48:34.701 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-24T02:48:34.716 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-03-24T02:48:34.821 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-24T02:48:34.864 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-24T02:48:34.881 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-24T02:48:34.917 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-24T02:48:34.940 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-24T02:48:34.949 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-24T02:48:35.002 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-24T02:48:35.057 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-24T02:48:35.098 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-24T02:48:35.101 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-24T02:48:35.123 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-24T02:48:35.131 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-24T02:48:35.168 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-24T02:48:35.211 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-24T02:48:35.368 INFO:teuthology.orchestra.run.smithi188.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 312 kB/s | 1.1 MB 00:03 2024-03-24T02:48:35.523 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-24T02:48:35.574 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-24T02:48:35.697 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-24T02:48:35.725 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-24T02:48:35.817 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-24T02:48:35.855 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-24T02:48:35.892 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-24T02:48:35.906 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-24T02:48:35.998 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-24T02:48:36.110 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-24T02:48:36.160 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-24T02:48:36.187 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-24T02:48:36.275 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-24T02:48:36.305 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-24T02:48:36.378 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-24T02:48:36.409 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-24T02:48:36.431 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-24T02:48:36.446 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-24T02:48:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-24T02:48:36.654 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-24T02:48:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-24T02:48:36.874 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-24T02:48:36.996 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-24T02:48:37.039 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-24T02:48:37.132 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-24T02:48:37.184 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-24T02:48:37.283 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-24T02:48:37.365 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-24T02:48:37.391 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-24T02:48:37.396 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-24T02:48:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-24T02:48:37.455 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-24T02:48:37.500 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-24T02:48:37.605 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-24T02:48:37.639 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-24T02:48:37.666 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-24T02:48:37.702 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-24T02:48:37.779 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-24T02:48:37.783 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-24T02:48:37.809 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-24T02:48:37.839 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-24T02:48:37.877 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-24T02:48:37.884 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-24T02:48:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-24T02:48:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-24T02:48:37.952 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-24T02:48:37.993 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-24T02:48:38.036 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-24T02:48:38.069 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-24T02:48:38.096 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-24T02:48:38.168 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-24T02:48:38.201 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-24T02:48:38.252 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-24T02:48:38.290 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-24T02:48:38.362 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-24T02:48:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-24T02:48:38.436 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-24T02:48:38.462 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-24T02:48:38.499 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-24T02:48:38.533 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-24T02:48:38.563 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-24T02:48:38.637 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-24T02:48:38.692 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-24T02:48:39.271 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-24T02:48:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-24T02:48:39.436 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-24T02:48:39.489 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-24T02:48:39.496 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-24T02:48:39.530 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-24T02:48:39.571 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-24T02:48:39.617 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-24T02:48:39.647 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-24T02:48:39.911 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-24T02:48:40.074 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-24T02:48:40.158 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-24T02:48:40.210 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-24T02:48:40.358 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-24T02:48:40.413 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-24T02:48:40.469 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-24T02:48:40.543 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-24T02:48:40.580 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-24T02:48:40.617 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-24T02:48:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-24T02:48:40.700 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-24T02:48:40.752 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-24T02:48:40.802 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-24T02:48:40.847 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-24T02:48:40.885 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-24T02:48:40.943 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-24T02:48:40.991 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-24T02:48:41.021 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-24T02:48:41.063 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-24T02:48:41.161 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-24T02:48:41.195 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-24T02:48:41.236 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-24T02:48:41.270 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-24T02:48:41.294 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-24T02:48:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-24T02:48:41.452 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-24T02:48:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-24T02:48:41.506 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-24T02:48:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-24T02:48:41.562 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-24T02:48:41.562 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-24T02:48:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:48:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-24T02:48:41.616 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-24T02:48:41.661 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-24T02:48:41.711 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-24T02:48:41.767 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-24T02:48:41.800 INFO:teuthology.orchestra.run.smithi188.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 215 kB/s | 2.3 MB 00:10 2024-03-24T02:48:41.805 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-03-24T02:48:41.805 INFO:teuthology.orchestra.run.smithi188.stdout:Total 6.3 MB/s | 224 MB 00:35 2024-03-24T02:48:41.966 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-24T02:48:42.016 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-24T02:48:42.158 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-24T02:48:42.205 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-24T02:48:42.237 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-24T02:48:42.268 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-24T02:48:42.286 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-03-24T02:48:42.296 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-24T02:48:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-24T02:48:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-24T02:48:42.435 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-24T02:48:42.465 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-24T02:48:42.543 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-24T02:48:42.558 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-03-24T02:48:42.558 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-03-24T02:48:42.575 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-24T02:48:42.617 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-24T02:48:42.667 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-24T02:48:42.695 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-24T02:48:42.797 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-24T02:48:42.905 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-24T02:48:42.952 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-24T02:48:42.982 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-24T02:48:43.030 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-24T02:48:43.131 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-24T02:48:43.185 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-24T02:48:43.949 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-24T02:48:44.160 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-24T02:48:44.257 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-24T02:48:44.314 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-24T02:48:44.331 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-03-24T02:48:44.332 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-03-24T02:48:44.404 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-24T02:48:44.441 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-24T02:48:44.625 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-24T02:48:44.664 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-24T02:48:44.750 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-24T02:48:45.442 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-24T02:48:45.563 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-24T02:48:46.844 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-24T02:48:46.923 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-24T02:48:46.963 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-24T02:48:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-24T02:48:47.029 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-24T02:48:47.358 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-24T02:48:47.384 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-24T02:48:47.475 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-24T02:48:47.514 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-24T02:48:47.551 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-24T02:48:48.035 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-24T02:48:48.062 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-24T02:48:48.125 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-24T02:48:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-24T02:48:50.453 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-24T02:48:50.488 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-24T02:48:51.067 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-24T02:48:51.099 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-24T02:48:51.179 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-24T02:48:51.180 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-24T02:48:51.180 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-24T02:48:51.180 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:48:51.217 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-24T02:48:51.308 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-03-24T02:48:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-24T02:48:51.454 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-24T02:48:51.485 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-24T02:48:51.494 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-24T02:48:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-24T02:48:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-24T02:48:51.622 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-24T02:48:51.656 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-24T02:48:51.698 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-24T02:48:52.029 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-24T02:48:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-24T02:48:52.212 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-24T02:48:52.241 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-24T02:48:52.317 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-24T02:48:52.346 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-24T02:48:52.387 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-24T02:48:52.572 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-24T02:48:52.627 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-24T02:48:52.655 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-24T02:48:52.742 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-24T02:48:52.765 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-24T02:48:52.812 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-24T02:48:52.868 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-24T02:48:52.891 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-24T02:48:52.907 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-24T02:48:53.075 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-24T02:48:53.106 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-24T02:48:53.137 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-24T02:48:53.235 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-24T02:48:53.348 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-24T02:48:53.409 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-24T02:48:53.484 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-24T02:48:53.535 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-24T02:48:53.747 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-24T02:48:53.862 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-24T02:48:53.876 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-24T02:48:53.876 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-03-24T02:48:53.877 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-03-24T02:48:53.877 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-03-24T02:48:53.877 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-03-24T02:48:53.877 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-03-24T02:48:53.877 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-03-24T02:48:53.877 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-03-24T02:48:53.877 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:48:53.953 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-24T02:48:53.990 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-24T02:48:54.017 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-24T02:48:54.046 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-24T02:48:54.052 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-24T02:48:54.062 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-24T02:48:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:48:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-24T02:48:54.062 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:48:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-24T02:48:54.181 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-24T02:48:54.219 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-24T02:48:54.284 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-24T02:48:54.318 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-24T02:48:54.370 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-24T02:48:54.403 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-24T02:48:54.431 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-24T02:48:54.500 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-24T02:48:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-24T02:48:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-24T02:48:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-24T02:48:54.675 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-24T02:48:54.722 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-24T02:48:54.794 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-24T02:48:54.937 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-24T02:48:54.992 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-24T02:48:55.513 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-24T02:48:55.582 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-24T02:48:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-24T02:48:55.609 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-24T02:48:55.609 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:48:55.610 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-24T02:48:55.610 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T02:48:55.610 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T02:48:55.610 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:48:55.674 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-24T02:48:55.741 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-24T02:48:55.749 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-24T02:48:55.780 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-24T02:48:55.829 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-24T02:48:55.850 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-24T02:48:55.867 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-24T02:48:55.874 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-24T02:48:55.874 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:48:55.874 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-24T02:48:55.874 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T02:48:55.874 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T02:48:55.874 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:48:55.897 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-24T02:48:56.120 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-24T02:48:56.302 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-24T02:48:56.334 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-24T02:48:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-24T02:48:56.358 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:48:56.358 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-24T02:48:56.358 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T02:48:56.358 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T02:48:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:48:56.362 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-24T02:48:56.406 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-24T02:48:56.452 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-24T02:48:56.532 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-24T02:48:56.624 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-24T02:48:56.682 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-24T02:48:56.761 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-24T02:48:56.803 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-24T02:48:56.839 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-24T02:48:56.847 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-24T02:48:56.867 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-24T02:48:56.885 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-24T02:48:56.900 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-24T02:48:56.921 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-24T02:48:56.961 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-24T02:48:56.964 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-24T02:48:56.990 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-24T02:48:57.007 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-24T02:48:57.139 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-24T02:48:57.196 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-24T02:48:57.226 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-24T02:48:57.237 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-24T02:48:57.266 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-24T02:48:57.267 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-24T02:48:57.369 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-24T02:48:57.409 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-24T02:48:57.449 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-24T02:48:57.475 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-24T02:48:57.500 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-24T02:48:57.572 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-24T02:48:57.709 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-24T02:48:57.815 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-24T02:48:57.867 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-24T02:48:57.915 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-24T02:48:57.970 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-24T02:48:58.157 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-24T02:48:58.205 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-24T02:48:58.280 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-24T02:48:58.317 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-24T02:48:58.351 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-24T02:48:58.395 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-24T02:48:58.427 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-24T02:48:58.427 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-24T02:48:58.458 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-24T02:48:58.486 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-24T02:48:58.520 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-24T02:48:58.559 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-24T02:48:58.586 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-24T02:48:58.590 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-24T02:48:58.610 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-24T02:48:58.634 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-24T02:48:58.634 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:48:58.634 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-24T02:48:58.635 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T02:48:58.635 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T02:48:58.635 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:48:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-24T02:48:58.686 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-24T02:48:58.697 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-24T02:48:58.730 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-24T02:48:58.736 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-24T02:48:58.782 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-24T02:48:58.815 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-24T02:48:58.856 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-24T02:48:58.891 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-24T02:48:58.980 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-24T02:48:59.070 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-24T02:48:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-24T02:48:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-24T02:48:59.201 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-24T02:48:59.298 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-24T02:48:59.364 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-24T02:48:59.658 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-24T02:48:59.688 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-24T02:48:59.712 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-24T02:48:59.712 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:48:59.713 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-24T02:48:59.713 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T02:48:59.713 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T02:48:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:48:59.755 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-24T02:48:59.778 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-24T02:48:59.779 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:48:59.779 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-24T02:48:59.779 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:48:59.822 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-24T02:48:59.846 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-24T02:48:59.846 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:48:59.846 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-24T02:48:59.846 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-24T02:48:59.847 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-24T02:48:59.847 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:49:00.091 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-24T02:49:00.144 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-24T02:49:00.165 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-24T02:49:00.178 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-24T02:49:00.189 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-24T02:49:00.189 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:00.189 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-24T02:49:00.189 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T02:49:00.189 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T02:49:00.189 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:49:00.226 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-24T02:49:00.315 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-24T02:49:00.346 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-24T02:49:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-24T02:49:00.560 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-24T02:49:00.637 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-24T02:49:01.285 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-24T02:49:01.390 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-24T02:49:02.595 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-24T02:49:02.687 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-24T02:49:02.718 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-24T02:49:02.749 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-24T02:49:02.777 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-24T02:49:03.098 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-24T02:49:03.123 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-24T02:49:03.197 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-24T02:49:03.228 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-24T02:49:03.242 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-24T02:49:03.242 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /sys 2024-03-24T02:49:03.242 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /proc 2024-03-24T02:49:03.242 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /mnt 2024-03-24T02:49:03.242 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /var/tmp 2024-03-24T02:49:03.242 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /home 2024-03-24T02:49:03.242 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /root 2024-03-24T02:49:03.242 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /tmp 2024-03-24T02:49:03.242 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:49:03.264 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-24T02:49:03.414 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-24T02:49:03.428 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-24T02:49:03.428 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:03.428 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-24T02:49:03.428 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:49:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-24T02:49:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-24T02:49:03.822 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-24T02:49:03.845 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-24T02:49:04.886 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-24T02:49:04.928 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-24T02:49:04.958 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-24T02:49:04.958 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-24T02:49:04.983 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-24T02:49:04.984 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:04.984 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-24T02:49:04.984 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T02:49:04.984 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T02:49:04.984 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:49:04.985 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-24T02:49:05.175 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-24T02:49:05.217 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-24T02:49:05.239 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-24T02:49:05.239 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:05.239 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-24T02:49:05.239 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T02:49:05.240 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T02:49:05.240 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:49:05.297 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-24T02:49:05.343 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-24T02:49:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-24T02:49:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-24T02:49:05.467 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-24T02:49:05.492 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-24T02:49:05.548 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-24T02:49:05.579 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-24T02:49:05.625 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-24T02:49:05.695 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-24T02:49:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-24T02:49:05.723 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-24T02:49:05.723 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:05.723 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-24T02:49:05.723 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T02:49:05.724 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T02:49:05.724 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:49:05.727 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-24T02:49:05.812 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-24T02:49:06.012 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-24T02:49:06.049 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-24T02:49:06.220 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-24T02:49:06.252 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-24T02:49:06.284 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-24T02:49:06.329 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-24T02:49:06.345 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-24T02:49:06.367 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-24T02:49:06.375 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-24T02:49:06.388 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-24T02:49:06.587 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-24T02:49:06.618 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-24T02:49:06.620 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-24T02:49:06.651 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-24T02:49:06.684 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-24T02:49:06.685 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-24T02:49:06.685 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-24T02:49:06.685 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:06.712 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-24T02:49:07.281 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-24T02:49:07.354 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-24T02:49:07.395 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-24T02:49:07.479 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-24T02:49:07.515 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-24T02:49:07.549 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-24T02:49:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-24T02:49:07.600 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-24T02:49:07.600 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-24T02:49:07.658 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-24T02:49:07.684 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-24T02:49:07.689 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-24T02:49:07.718 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-24T02:49:07.718 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-24T02:49:07.787 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-24T02:49:07.795 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-24T02:49:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-24T02:49:07.854 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-24T02:49:07.888 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-24T02:49:07.912 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-24T02:49:07.946 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-24T02:49:07.963 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-24T02:49:07.985 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-24T02:49:07.995 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-24T02:49:08.017 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-24T02:49:08.017 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-24T02:49:08.017 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:08.018 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-24T02:49:08.018 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T02:49:08.018 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T02:49:08.018 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:49:08.042 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-24T02:49:08.063 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-24T02:49:08.068 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-24T02:49:08.073 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-24T02:49:08.093 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-24T02:49:08.118 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-24T02:49:08.126 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-24T02:49:08.126 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-24T02:49:08.126 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-24T02:49:08.313 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-24T02:49:08.313 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-24T02:49:08.388 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-24T02:49:08.412 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-24T02:49:08.444 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-24T02:49:08.471 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-24T02:49:08.496 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-24T02:49:08.564 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-24T02:49:09.046 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-24T02:49:09.073 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-24T02:49:09.095 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-24T02:49:09.095 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:09.096 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-24T02:49:09.096 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T02:49:09.096 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T02:49:09.096 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:49:09.140 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-24T02:49:09.162 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-24T02:49:09.162 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:09.162 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-24T02:49:09.162 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:49:09.208 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-24T02:49:09.229 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-24T02:49:09.230 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:09.230 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-24T02:49:09.230 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-24T02:49:09.230 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-24T02:49:09.230 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:49:09.550 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-24T02:49:09.572 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-24T02:49:09.572 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:09.572 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-24T02:49:09.572 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T02:49:09.572 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T02:49:09.572 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:49:13.389 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-24T02:49:13.390 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-24T02:49:13.390 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-24T02:49:13.390 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-24T02:49:13.390 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-24T02:49:13.390 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-24T02:49:13.390 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-24T02:49:13.390 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-24T02:49:13.390 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-24T02:49:13.390 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-24T02:49:13.390 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-24T02:49:13.390 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-24T02:49:13.390 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-24T02:49:13.391 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-24T02:49:13.392 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-24T02:49:13.393 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-24T02:49:13.394 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-24T02:49:13.395 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-24T02:49:13.396 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-24T02:49:13.396 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-24T02:49:13.396 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-24T02:49:13.396 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-24T02:49:13.396 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-24T02:49:13.396 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-24T02:49:13.396 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-24T02:49:13.396 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-24T02:49:13.396 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-24T02:49:13.396 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-24T02:49:13.396 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-24T02:49:13.397 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-24T02:49:13.397 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-24T02:49:13.397 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-24T02:49:13.397 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-24T02:49:13.397 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-24T02:49:13.397 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-24T02:49:13.397 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-24T02:49:13.397 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-24T02:49:13.397 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-24T02:49:13.397 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-24T02:49:13.397 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-24T02:49:13.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-24T02:49:13.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-24T02:49:13.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-24T02:49:13.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-24T02:49:13.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-24T02:49:13.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-24T02:49:13.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-24T02:49:13.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-24T02:49:13.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-24T02:49:13.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-24T02:49:13.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-24T02:49:13.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-24T02:49:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-24T02:49:13.400 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-24T02:49:13.401 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-24T02:49:13.402 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-24T02:49:13.402 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-24T02:49:13.402 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-24T02:49:13.402 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-24T02:49:13.402 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-24T02:49:13.402 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-24T02:49:13.402 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-24T02:49:13.402 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-24T02:49:13.402 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-24T02:49:13.402 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-24T02:49:13.402 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-24T02:49:13.402 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-24T02:49:13.403 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-24T02:49:13.403 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-24T02:49:13.403 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-24T02:49:13.403 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-24T02:49:13.403 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-24T02:49:13.403 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-24T02:49:13.403 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-24T02:49:13.403 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-24T02:49:13.403 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-24T02:49:13.403 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-24T02:49:13.403 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-24T02:49:13.404 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-24T02:49:13.404 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-24T02:49:13.404 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-24T02:49:13.404 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-24T02:49:13.404 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-24T02:49:13.404 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-24T02:49:13.404 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-24T02:49:13.404 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-24T02:49:13.404 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-24T02:49:13.404 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-24T02:49:13.404 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-24T02:49:13.405 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-24T02:49:13.405 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-24T02:49:13.405 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-24T02:49:13.405 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-24T02:49:13.405 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-24T02:49:14.305 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-24T02:49:14.346 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-24T02:49:14.376 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-24T02:49:14.404 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-24T02:49:14.603 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-24T02:49:14.732 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-24T02:49:14.778 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-24T02:49:14.834 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-24T02:49:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-24T02:49:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-24T02:49:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-24T02:49:15.010 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-24T02:49:15.040 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-24T02:49:15.111 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-24T02:49:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-24T02:49:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-24T02:49:15.796 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-24T02:49:15.839 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-24T02:49:15.857 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-24T02:49:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-24T02:49:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:49:16.307 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-03-24T02:49:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-24T02:49:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-24T02:49:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-24T02:49:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: acl-2.3.1-4.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-24T02:49:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-24T02:49:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: jq-1.6-16.el9.x86_64 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-24T02:49:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-24T02:49:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.11-1.el9.x86_64 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-24T02:49:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-24T02:49:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-24T02:49:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-24T02:49:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-24T02:49:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-24T02:49:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-24T02:49:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-24T02:49:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-24T02:49:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-24T02:49:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-24T02:49:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-24T02:49:16.320 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-24T02:49:16.726 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-24T02:49:16.804 DEBUG:teuthology.parallel:result is None 2024-03-24T02:49:16.804 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-24T02:49:16.842 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-24T02:49:16.925 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-24T02:49:16.954 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-24T02:49:16.979 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-24T02:49:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-24T02:49:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-24T02:49:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-24T02:49:17.119 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-24T02:49:17.157 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-24T02:49:17.157 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-24T02:49:17.242 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-24T02:49:17.268 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-24T02:49:17.301 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-24T02:49:17.326 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-24T02:49:17.352 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-24T02:49:17.386 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-24T02:49:17.424 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-24T02:49:17.456 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-24T02:49:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-24T02:49:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-24T02:49:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-24T02:49:17.566 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-24T02:49:17.572 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-24T02:49:17.572 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-24T02:49:17.761 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-24T02:49:17.761 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-24T02:49:17.843 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-24T02:49:17.868 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-24T02:49:17.901 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-24T02:49:17.928 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-24T02:49:17.953 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-24T02:49:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-24T02:49:18.334 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-24T02:49:18.334 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-03-24T02:49:18.334 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-03-24T02:49:18.334 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-03-24T02:49:18.334 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-03-24T02:49:18.334 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-03-24T02:49:18.334 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-03-24T02:49:18.334 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-03-24T02:49:18.334 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:18.512 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-24T02:49:18.530 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-24T02:49:18.530 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:18.530 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-24T02:49:18.530 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:20.023 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-24T02:49:20.049 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-24T02:49:20.049 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:20.049 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-24T02:49:20.049 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T02:49:20.050 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T02:49:20.050 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:20.274 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-24T02:49:20.296 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-24T02:49:20.296 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:20.296 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-24T02:49:20.297 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T02:49:20.297 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T02:49:20.297 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:20.749 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-24T02:49:20.772 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-24T02:49:20.772 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:20.772 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-24T02:49:20.772 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T02:49:20.772 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T02:49:20.772 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:20.868 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-24T02:49:21.294 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-24T02:49:21.325 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-24T02:49:21.357 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-24T02:49:21.416 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-24T02:49:21.448 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-24T02:49:21.682 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-24T02:49:21.716 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-24T02:49:22.675 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-24T02:49:22.700 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-24T02:49:22.802 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-24T02:49:22.832 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-24T02:49:22.832 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-24T02:49:22.832 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-24T02:49:22.833 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-24T02:49:22.833 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-24T02:49:22.833 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-24T02:49:22.833 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-24T02:49:22.833 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-24T02:49:22.833 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-24T02:49:22.834 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-24T02:49:22.834 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-24T02:49:22.834 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-24T02:49:22.834 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-24T02:49:22.834 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-24T02:49:22.834 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-24T02:49:22.835 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-24T02:49:22.835 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-24T02:49:22.835 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-24T02:49:22.835 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-24T02:49:22.835 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-24T02:49:22.835 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-24T02:49:22.835 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-24T02:49:22.836 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-24T02:49:22.836 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-24T02:49:22.836 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-24T02:49:22.836 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-24T02:49:22.836 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-24T02:49:22.836 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-24T02:49:22.837 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-24T02:49:22.837 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-24T02:49:22.837 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-24T02:49:22.837 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-24T02:49:22.837 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-24T02:49:22.837 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-24T02:49:22.838 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-24T02:49:22.838 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-24T02:49:22.838 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-24T02:49:22.838 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-24T02:49:22.838 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-24T02:49:22.838 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-24T02:49:22.838 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-24T02:49:22.839 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-24T02:49:22.839 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-24T02:49:22.839 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-24T02:49:22.839 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-24T02:49:22.839 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-24T02:49:22.839 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-24T02:49:22.839 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-24T02:49:22.840 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-24T02:49:22.840 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-24T02:49:22.840 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-24T02:49:22.840 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-24T02:49:22.840 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-24T02:49:22.841 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-24T02:49:22.841 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-24T02:49:22.841 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-24T02:49:22.841 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-24T02:49:22.841 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-24T02:49:22.842 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-24T02:49:22.842 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-24T02:49:22.842 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-24T02:49:22.842 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-24T02:49:22.843 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-24T02:49:22.843 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-24T02:49:22.843 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-24T02:49:22.843 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-24T02:49:22.843 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-24T02:49:22.844 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-24T02:49:22.844 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-24T02:49:22.844 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-24T02:49:22.844 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-24T02:49:22.844 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-24T02:49:22.845 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-24T02:49:22.845 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-24T02:49:22.845 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-24T02:49:22.845 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-24T02:49:22.845 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-24T02:49:22.846 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-24T02:49:22.846 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-24T02:49:22.846 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-24T02:49:22.846 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-24T02:49:22.846 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-24T02:49:22.847 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-24T02:49:22.847 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-24T02:49:22.847 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-24T02:49:22.847 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-24T02:49:22.847 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-24T02:49:22.848 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-24T02:49:22.848 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-24T02:49:22.848 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-24T02:49:22.848 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-24T02:49:22.848 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-24T02:49:22.849 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-24T02:49:22.849 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-24T02:49:22.849 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-24T02:49:22.849 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-24T02:49:22.849 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-24T02:49:22.850 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-24T02:49:22.850 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-24T02:49:22.850 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-24T02:49:22.850 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-24T02:49:22.850 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-24T02:49:22.851 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-24T02:49:22.851 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-24T02:49:22.851 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-24T02:49:22.851 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-24T02:49:22.851 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-24T02:49:22.852 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-24T02:49:22.852 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-24T02:49:22.852 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-24T02:49:22.852 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-24T02:49:22.852 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-24T02:49:22.853 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-24T02:49:22.853 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-24T02:49:22.853 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-24T02:49:22.853 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-24T02:49:22.853 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-24T02:49:22.854 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-24T02:49:22.854 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-24T02:49:22.854 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-24T02:49:22.854 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-24T02:49:22.854 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-24T02:49:22.855 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-24T02:49:22.855 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-24T02:49:22.855 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-24T02:49:22.855 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-24T02:49:22.855 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-24T02:49:22.856 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-24T02:49:22.856 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-24T02:49:22.856 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-24T02:49:22.856 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-24T02:49:22.856 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-24T02:49:22.857 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-24T02:49:22.857 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-24T02:49:22.857 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-24T02:49:22.857 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-24T02:49:22.857 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-24T02:49:22.857 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-24T02:49:22.857 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-24T02:49:22.857 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-24T02:49:22.857 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-24T02:49:22.857 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-24T02:49:22.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-24T02:49:22.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-24T02:49:22.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-24T02:49:22.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-24T02:49:22.862 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-24T02:49:22.954 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-24T02:49:22.985 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-24T02:49:23.008 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-24T02:49:23.008 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:23.008 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-24T02:49:23.008 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T02:49:23.008 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T02:49:23.008 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:23.052 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-24T02:49:23.063 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-24T02:49:23.108 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-24T02:49:23.982 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-24T02:49:24.021 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-24T02:49:24.044 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-24T02:49:24.044 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:24.044 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-24T02:49:24.044 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T02:49:24.044 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T02:49:24.045 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:24.089 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-24T02:49:24.110 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-24T02:49:24.110 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:24.110 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-24T02:49:24.110 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:24.156 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-24T02:49:24.178 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-24T02:49:24.178 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:24.178 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-24T02:49:24.179 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-24T02:49:24.179 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-24T02:49:24.179 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:24.498 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-24T02:49:24.521 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-24T02:49:24.521 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T02:49:24.521 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-24T02:49:24.521 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T02:49:24.521 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T02:49:24.521 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:25.588 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-24T02:49:25.588 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:49:25.588 INFO:teuthology.orchestra.run.smithi017.stdout:Upgraded: 2024-03-24T02:49:25.588 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: acl-2.3.1-4.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.589 INFO:teuthology.orchestra.run.smithi017.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout: automake-1.16.2-8.el9.noarch 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout: bison-3.7.4-5.el9.x86_64 2024-03-24T02:49:25.590 INFO:teuthology.orchestra.run.smithi017.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:25.591 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: flex-2.6.4-9.el9.x86_64 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-24T02:49:25.592 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: jq-1.6-16.el9.x86_64 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-24T02:49:25.593 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: lua-5.4.4-4.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-24T02:49:25.594 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-13.11-1.el9.x86_64 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-24T02:49:25.595 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-24T02:49:25.596 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-24T02:49:25.597 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ply-3.11-14.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.598 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-24T02:49:25.599 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: re2-1:20211101-3.el9.x86_64 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-24T02:49:25.600 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-24T02:49:25.601 INFO:teuthology.orchestra.run.smithi017.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-24T02:49:25.601 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:49:25.601 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-03-24T02:49:26.063 DEBUG:teuthology.parallel:result is None 2024-03-24T02:49:28.961 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-24T02:49:29.002 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-24T02:49:29.033 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-24T02:49:29.060 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-24T02:49:29.240 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-24T02:49:29.351 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-24T02:49:29.391 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-24T02:49:29.431 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-24T02:49:29.492 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-24T02:49:29.535 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-24T02:49:29.560 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-24T02:49:29.583 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-24T02:49:29.613 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-24T02:49:29.670 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-24T02:49:29.726 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-24T02:49:29.762 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-24T02:49:30.273 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-24T02:49:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-24T02:49:30.339 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-24T02:49:31.154 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-24T02:49:31.223 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-24T02:49:31.263 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-24T02:49:31.345 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-24T02:49:31.385 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-24T02:49:31.410 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-24T02:49:31.444 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-24T02:49:31.479 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-24T02:49:31.479 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-24T02:49:31.566 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-24T02:49:31.596 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-24T02:49:31.596 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-24T02:49:31.673 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-24T02:49:31.698 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-24T02:49:31.732 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-24T02:49:31.766 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-24T02:49:31.791 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-24T02:49:31.823 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-24T02:49:31.863 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-24T02:49:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-24T02:49:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-24T02:49:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-24T02:49:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-24T02:49:32.005 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-24T02:49:32.011 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-24T02:49:32.012 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-24T02:49:32.192 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-24T02:49:32.192 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-24T02:49:32.274 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-24T02:49:32.299 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-24T02:49:32.332 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-24T02:49:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-24T02:49:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-24T02:49:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-24T02:49:37.539 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-24T02:49:37.539 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-24T02:49:37.539 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-24T02:49:37.539 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-24T02:49:37.539 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-24T02:49:37.539 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-24T02:49:37.539 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-24T02:49:37.540 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-24T02:49:37.541 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-24T02:49:37.542 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-24T02:49:37.543 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-24T02:49:37.544 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-24T02:49:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-24T02:49:37.546 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-24T02:49:37.547 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-24T02:49:37.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-24T02:49:37.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-24T02:49:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-24T02:49:37.551 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-24T02:49:37.552 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-24T02:49:37.553 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-24T02:49:37.553 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-24T02:49:37.553 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-24T02:49:37.553 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-24T02:49:37.553 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-24T02:49:37.553 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-24T02:49:37.553 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-24T02:49:37.553 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-24T02:49:37.553 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-24T02:49:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-24T02:49:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:40.444 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-03-24T02:49:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: acl-2.3.1-4.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-24T02:49:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-03-24T02:49:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-03-24T02:49:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: jq-1.6-16.el9.x86_64 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-24T02:49:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-03-24T02:49:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.11-1.el9.x86_64 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-24T02:49:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-24T02:49:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-24T02:49:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-24T02:49:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-24T02:49:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-24T02:49:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-03-24T02:49:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-24T02:49:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-24T02:49:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-24T02:49:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-24T02:49:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-24T02:49:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-24T02:49:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:49:40.457 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-03-24T02:49:40.941 DEBUG:teuthology.parallel:result is None 2024-03-24T02:49:40.942 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T02:49:41.085 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-24T02:49:41.110 INFO:teuthology.orchestra.run.smithi017.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-24T02:49:41.111 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-24T02:49:41.111 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-24T02:49:41.113 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T02:49:41.325 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-24T02:49:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-24T02:49:41.352 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-24T02:49:41.352 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-24T02:49:41.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T02:49:41.489 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-24T02:49:41.511 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-24T02:49:41.512 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-24T02:49:41.512 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-24T02:49:41.513 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-24T02:49:41.514 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:49:41.514 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-24T02:49:41.548 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-24T02:49:41.548 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-24T02:49:41.587 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-24T02:49:41.587 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-24T02:49:41.617 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-24T02:49:41.618 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:49:41.618 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/daemon-helper 2024-03-24T02:49:41.648 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-24T02:49:41.718 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-24T02:49:41.718 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-03-24T02:49:41.753 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-24T02:49:41.828 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-24T02:49:41.828 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-03-24T02:49:41.857 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-24T02:49:41.925 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-24T02:49:41.926 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:49:41.926 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-24T02:49:41.956 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-24T02:49:42.026 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-24T02:49:42.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-24T02:49:42.063 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-24T02:49:42.135 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-24T02:49:42.135 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-24T02:49:42.164 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-24T02:49:42.233 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-24T02:49:42.233 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:49:42.233 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/stdin-killer 2024-03-24T02:49:42.264 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-24T02:49:42.334 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-24T02:49:42.334 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-03-24T02:49:42.364 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-24T02:49:42.435 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-24T02:49:42.435 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-03-24T02:49:42.465 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-24T02:49:42.534 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-24T02:49:42.631 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-24T02:49:42.632 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T02:49:42.632 INFO:tasks.cephadm:Cluster fsid is 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:49:42.632 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-24T02:49:42.632 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.17', 'mon.b': '172.21.15.19', 'mon.c': '172.21.15.188'} 2024-03-24T02:49:42.632 INFO:tasks.cephadm:First mon is mon.a on smithi017 2024-03-24T02:49:42.633 INFO:tasks.cephadm:First mgr is x 2024-03-24T02:49:42.633 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-24T02:49:42.633 DEBUG:teuthology.orchestra.run.smithi017:> sudo hostname $(hostname -s) 2024-03-24T02:49:42.664 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-03-24T02:49:42.695 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname $(hostname -s) 2024-03-24T02:49:42.725 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-24T02:49:42.725 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T02:49:42.863 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77877/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2145-g3be9a5c9', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2145.g3be9a5c9'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-22 05:51:42.267384', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-main-20240318.032620', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-24T02:49:43.076 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-mchangir-testing-main-20240318.032620, sha1 3be9a5c9bc793e11e2800a8c0c696e8b46742033 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T02:49:43.077 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm 2024-03-24T02:49:43.078 DEBUG:teuthology.orchestra.run.smithi017:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-24T02:49:43.533 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 24 02:49 /home/ubuntu/cephtest/cephadm 2024-03-24T02:49:43.533 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-24T02:49:43.970 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 24 02:49 /home/ubuntu/cephtest/cephadm 2024-03-24T02:49:43.970 DEBUG:teuthology.orchestra.run.smithi188:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-24T02:49:44.394 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 24 02:49 /home/ubuntu/cephtest/cephadm 2024-03-24T02:49:44.395 DEBUG:teuthology.orchestra.run.smithi017:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-24T02:49:44.413 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-24T02:49:44.433 DEBUG:teuthology.orchestra.run.smithi188:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-24T02:49:44.467 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 on all hosts... 2024-03-24T02:49:44.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-24T02:49:44.469 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-24T02:49:44.476 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-24T02:49:45.328 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-24T02:49:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-24T02:49:45.421 INFO:teuthology.orchestra.run.smithi188.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-24T02:50:15.464 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T02:50:15.464 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-24T02:50:15.464 INFO:teuthology.orchestra.run.smithi017.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-24T02:50:15.464 INFO:teuthology.orchestra.run.smithi017.stdout: "repo_digests": [ 2024-03-24T02:50:15.464 INFO:teuthology.orchestra.run.smithi017.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-24T02:50:15.464 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-24T02:50:15.464 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T02:50:19.978 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-03-24T02:50:19.979 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-24T02:50:19.979 INFO:teuthology.orchestra.run.smithi188.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-24T02:50:19.979 INFO:teuthology.orchestra.run.smithi188.stdout: "repo_digests": [ 2024-03-24T02:50:19.979 INFO:teuthology.orchestra.run.smithi188.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-24T02:50:19.979 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-03-24T02:50:19.979 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-03-24T02:50:22.421 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-24T02:50:22.421 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-24T02:50:22.421 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-24T02:50:22.421 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-03-24T02:50:22.422 INFO:teuthology.orchestra.run.smithi019.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-24T02:50:22.422 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-03-24T02:50:22.422 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-24T02:50:22.447 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph 2024-03-24T02:50:22.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-03-24T02:50:22.517 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph 2024-03-24T02:50:22.552 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 777 /etc/ceph 2024-03-24T02:50:22.581 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-03-24T02:50:22.612 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /etc/ceph 2024-03-24T02:50:22.642 INFO:tasks.cephadm:Writing seed config... 2024-03-24T02:50:22.643 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-24T02:50:22.643 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-24T02:50:22.643 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-24T02:50:22.643 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-24T02:50:22.643 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-24T02:50:22.644 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-24T02:50:22.645 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-24T02:50:22.646 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-24T02:50:22.646 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-24T02:50:22.646 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-24T02:50:22.646 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-24T02:50:22.646 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-24T02:50:22.646 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-24T02:50:22.646 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-24T02:50:22.646 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-24T02:50:22.646 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-24T02:50:22.646 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-24T02:50:22.646 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-24T02:50:22.647 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:50:22.647 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-24T02:50:22.663 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2a682800-e989-11ee-95cd-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-24T02:50:22.663 DEBUG:teuthology.orchestra.run.smithi017:mon.a> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.a.service 2024-03-24T02:50:22.706 DEBUG:teuthology.orchestra.run.smithi017:mgr.x> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.x.service 2024-03-24T02:50:22.749 INFO:tasks.cephadm:Bootstrapping... 2024-03-24T02:50:22.750 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 -v bootstrap --fsid 2a682800-e989-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.17 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-24T02:50:22.952 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-03-24T02:50:22.952 INFO:teuthology.orchestra.run.smithi017.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033', '-v', 'bootstrap', '--fsid', '2a682800-e989-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.17', '--skip-admin-label'] 2024-03-24T02:50:22.976 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-24T02:50:22.976 INFO:teuthology.orchestra.run.smithi017.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-24T02:50:22.976 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying podman|docker is present... 2024-03-24T02:50:23.000 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-24T02:50:23.000 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying lvm2 is present... 2024-03-24T02:50:23.000 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying time synchronization is in place... 2024-03-24T02:50:23.008 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-24T02:50:23.008 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-24T02:50:23.013 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-24T02:50:23.013 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2024-03-24T02:50:23.020 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout enabled 2024-03-24T02:50:23.026 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout active 2024-03-24T02:50:23.027 INFO:teuthology.orchestra.run.smithi017.stdout:Unit chronyd.service is enabled and running 2024-03-24T02:50:23.027 INFO:teuthology.orchestra.run.smithi017.stdout:Repeating the final host check... 2024-03-24T02:50:23.050 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-24T02:50:23.050 INFO:teuthology.orchestra.run.smithi017.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-24T02:50:23.050 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl is present 2024-03-24T02:50:23.050 INFO:teuthology.orchestra.run.smithi017.stdout:lvcreate is present 2024-03-24T02:50:23.056 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-24T02:50:23.056 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-24T02:50:23.062 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-24T02:50:23.062 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2024-03-24T02:50:23.068 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout enabled 2024-03-24T02:50:23.075 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout active 2024-03-24T02:50:23.075 INFO:teuthology.orchestra.run.smithi017.stdout:Unit chronyd.service is enabled and running 2024-03-24T02:50:23.075 INFO:teuthology.orchestra.run.smithi017.stdout:Host looks OK 2024-03-24T02:50:23.075 INFO:teuthology.orchestra.run.smithi017.stdout:Cluster fsid: 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:50:23.075 INFO:teuthology.orchestra.run.smithi017.stdout:Acquiring lock 140605320924608 on /run/cephadm/2a682800-e989-11ee-95cd-87774f69a715.lock 2024-03-24T02:50:23.075 INFO:teuthology.orchestra.run.smithi017.stdout:Lock 140605320924608 acquired on /run/cephadm/2a682800-e989-11ee-95cd-87774f69a715.lock 2024-03-24T02:50:23.076 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying IP 172.21.15.17 port 3300 ... 2024-03-24T02:50:23.076 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying IP 172.21.15.17 port 6789 ... 2024-03-24T02:50:23.076 INFO:teuthology.orchestra.run.smithi017.stdout:Base mon IP(s) is [172.21.15.17:3300, 172.21.15.17:6789], mon addrv is [v2:172.21.15.17:3300,v1:172.21.15.17:6789] 2024-03-24T02:50:23.079 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.17 metric 100 2024-03-24T02:50:23.079 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.17 metric 100 2024-03-24T02:50:23.081 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-24T02:50:23.081 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-24T02:50:23.081 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-24T02:50:23.084 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-24T02:50:23.084 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-24T02:50:23.084 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-24T02:50:23.084 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-24T02:50:23.084 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe59:d992/64 scope link noprefixroute 2024-03-24T02:50:23.084 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-24T02:50:23.085 INFO:teuthology.orchestra.run.smithi017.stdout:Mon IP `172.21.15.17` is in CIDR network `172.21.0.0/20` 2024-03-24T02:50:23.085 INFO:teuthology.orchestra.run.smithi017.stdout:Mon IP `172.21.15.17` is in CIDR network `172.21.0.0/20` 2024-03-24T02:50:23.085 INFO:teuthology.orchestra.run.smithi017.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-24T02:50:23.085 INFO:teuthology.orchestra.run.smithi017.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-24T02:50:23.086 INFO:teuthology.orchestra.run.smithi017.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-24T02:50:26.851 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stdout c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-24T02:50:26.851 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-24T02:50:26.852 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Getting image source signatures 2024-03-24T02:50:26.852 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Copying blob sha256:4dd83c6ee8921fe17396d77c31e205985ae94c87d3f752465b5f04aaf9ae340a 2024-03-24T02:50:26.852 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-24T02:50:26.852 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Copying config sha256:c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-24T02:50:26.852 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-24T02:50:27.889 INFO:teuthology.orchestra.run.smithi017.stdout:ceph: stdout ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-24T02:50:27.889 INFO:teuthology.orchestra.run.smithi017.stdout:Ceph version: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-24T02:50:27.890 INFO:teuthology.orchestra.run.smithi017.stdout:Extracting ceph user uid/gid from container image... 2024-03-24T02:50:28.978 INFO:teuthology.orchestra.run.smithi017.stdout:stat: stdout 167 167 2024-03-24T02:50:28.979 INFO:teuthology.orchestra.run.smithi017.stdout:Creating initial keys... 2024-03-24T02:50:29.833 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph-authtool: stdout AQD1lP9lPea3ExAAzcUSaA3wky6DhkJjX1ckqA== 2024-03-24T02:50:30.813 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph-authtool: stdout AQD2lP9lsz0SDRAAmOPsbH0RbQa0m5lcRtleLA== 2024-03-24T02:50:31.744 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph-authtool: stdout AQD3lP9leWlkCBAA29O4CFDoxGZdGE49O53bsA== 2024-03-24T02:50:31.745 INFO:teuthology.orchestra.run.smithi017.stdout:Creating initial monmap... 2024-03-24T02:50:32.749 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-24T02:50:32.750 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-24T02:50:32.750 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:50:32.750 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-24T02:50:32.750 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool for a [v2:172.21.15.17:3300,v1:172.21.15.17:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-24T02:50:32.750 INFO:teuthology.orchestra.run.smithi017.stdout:setting min_mon_release = quincy 2024-03-24T02:50:32.750 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: set fsid to 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:50:32.750 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-24T02:50:32.750 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:50:32.750 INFO:teuthology.orchestra.run.smithi017.stdout:Creating mon... 2024-03-24T02:50:33.680 INFO:teuthology.orchestra.run.smithi017.stdout:create mon.a on 2024-03-24T02:50:33.887 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-24T02:50:34.029 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-24T02:50:34.218 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2a682800-e989-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-2a682800-e989-11ee-95cd-87774f69a715.target. 2024-03-24T02:50:34.218 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2a682800-e989-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-2a682800-e989-11ee-95cd-87774f69a715.target. 2024-03-24T02:50:34.458 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.a 2024-03-24T02:50:34.458 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-24T02:50:34.629 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2a682800-e989-11ee-95cd-87774f69a715.target.wants/ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-2a682800-e989-11ee-95cd-87774f69a715@.service. 2024-03-24T02:50:35.172 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T02:50:35.172 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout disabled 2024-03-24T02:50:35.181 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T02:50:35.181 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2024-03-24T02:50:35.181 INFO:teuthology.orchestra.run.smithi017.stdout:firewalld.service is not enabled 2024-03-24T02:50:35.181 INFO:teuthology.orchestra.run.smithi017.stdout:Not possible to enable service . firewalld.service is not available 2024-03-24T02:50:35.181 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mon to start... 2024-03-24T02:50:35.181 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mon... 2024-03-24T02:50:36.246 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:36 smithi017 ceph-mon[31551]: from='client.? 172.21.15.17:0/1889264166' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-24T02:50:36.257 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout cluster: 2024-03-24T02:50:36.258 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout id: 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:50:36.258 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-24T02:50:36.258 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:36.258 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout services: 2024-03-24T02:50:36.258 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.540601s) 2024-03-24T02:50:36.258 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-24T02:50:36.258 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-24T02:50:36.258 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:36.258 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout data: 2024-03-24T02:50:36.258 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-24T02:50:36.258 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-24T02:50:36.259 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-24T02:50:36.259 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout pgs: 2024-03-24T02:50:36.259 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:36.259 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.611+0000 7f679e9b3700 1 Processor -- start 2024-03-24T02:50:36.259 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.611+0000 7f679e9b3700 1 -- start start 2024-03-24T02:50:36.259 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.611+0000 7f679e9b3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 0x7f6798105c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:36.259 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.611+0000 7f679e9b3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67981061a0 con 0x7f6798105840 2024-03-24T02:50:36.259 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.611+0000 7f6797fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 0x7f6798105c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:36.259 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.611+0000 7f6797fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 0x7f6798105c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39320/0 (socket says 172.21.15.17:39320) 2024-03-24T02:50:36.259 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.612+0000 7f6797fff700 1 -- 172.21.15.17:0/947781992 learned_addr learned my addr 172.21.15.17:0/947781992 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:36.259 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.612+0000 7f6797fff700 1 -- 172.21.15.17:0/947781992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798106320 con 0x7f6798105840 2024-03-24T02:50:36.260 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.612+0000 7f6797fff700 1 --2- 172.21.15.17:0/947781992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 0x7f6798105c60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f67880098a0 tx=0x7f6788009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309edb5c582c7369 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:36.260 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.612+0000 7f6796ffd700 1 -- 172.21.15.17:0/947781992 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6788004160 con 0x7f6798105840 2024-03-24T02:50:36.260 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.612+0000 7f6796ffd700 1 -- 172.21.15.17:0/947781992 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6788004300 con 0x7f6798105840 2024-03-24T02:50:36.260 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.612+0000 7f6796ffd700 1 -- 172.21.15.17:0/947781992 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67880036a0 con 0x7f6798105840 2024-03-24T02:50:36.260 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.613+0000 7f679e9b3700 1 -- 172.21.15.17:0/947781992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 msgr2=0x7f6798105c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:36.260 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.613+0000 7f679e9b3700 1 --2- 172.21.15.17:0/947781992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 0x7f6798105c60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f67880098a0 tx=0x7f6788009bb0 comp rx=0 tx=0).stop 2024-03-24T02:50:36.260 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.613+0000 7f679e9b3700 1 -- 172.21.15.17:0/947781992 shutdown_connections 2024-03-24T02:50:36.260 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.613+0000 7f679e9b3700 1 --2- 172.21.15.17:0/947781992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 0x7f6798105c60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:36.260 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.613+0000 7f679e9b3700 1 -- 172.21.15.17:0/947781992 >> 172.21.15.17:0/947781992 conn(0x7f6798101300 msgr2=0x7f6798103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:36.260 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.613+0000 7f679e9b3700 1 -- 172.21.15.17:0/947781992 shutdown_connections 2024-03-24T02:50:36.260 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.613+0000 7f679e9b3700 1 -- 172.21.15.17:0/947781992 wait complete. 2024-03-24T02:50:36.261 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.613+0000 7f679e9b3700 1 Processor -- start 2024-03-24T02:50:36.261 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.613+0000 7f679e9b3700 1 -- start start 2024-03-24T02:50:36.261 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.614+0000 7f679e9b3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 0x7f679819b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:36.261 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.614+0000 7f679e9b3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679810b320 con 0x7f6798105840 2024-03-24T02:50:36.261 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.614+0000 7f6797fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 0x7f679819b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:36.261 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.614+0000 7f6797fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 0x7f679819b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39324/0 (socket says 172.21.15.17:39324) 2024-03-24T02:50:36.261 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.614+0000 7f6797fff700 1 -- 172.21.15.17:0/1889264166 learned_addr learned my addr 172.21.15.17:0/1889264166 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:36.261 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.614+0000 7f6797fff700 1 -- 172.21.15.17:0/1889264166 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679819b710 con 0x7f6798105840 2024-03-24T02:50:36.261 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.614+0000 7f6797fff700 1 --2- 172.21.15.17:0/1889264166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 0x7f679819b1d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6788009fb0 tx=0x7f6788004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:36.261 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.614+0000 7f67957fa700 1 -- 172.21.15.17:0/1889264166 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6788003de0 con 0x7f6798105840 2024-03-24T02:50:36.261 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.614+0000 7f67957fa700 1 -- 172.21.15.17:0/1889264166 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6788011630 con 0x7f6798105840 2024-03-24T02:50:36.262 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.614+0000 7f679e9b3700 1 -- 172.21.15.17:0/1889264166 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679819b9a0 con 0x7f6798105840 2024-03-24T02:50:36.262 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.614+0000 7f67957fa700 1 -- 172.21.15.17:0/1889264166 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6788011930 con 0x7f6798105840 2024-03-24T02:50:36.262 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.614+0000 7f679e9b3700 1 -- 172.21.15.17:0/1889264166 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679819e690 con 0x7f6798105840 2024-03-24T02:50:36.262 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.615+0000 7f67957fa700 1 -- 172.21.15.17:0/1889264166 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6788011ad0 con 0x7f6798105840 2024-03-24T02:50:36.262 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.615+0000 7f67957fa700 1 -- 172.21.15.17:0/1889264166 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f678801b450 con 0x7f6798105840 2024-03-24T02:50:36.262 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.615+0000 7f679e9b3700 1 -- 172.21.15.17:0/1889264166 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67980637c0 con 0x7f6798105840 2024-03-24T02:50:36.262 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.618+0000 7f67957fa700 1 -- 172.21.15.17:0/1889264166 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6788004260 con 0x7f6798105840 2024-03-24T02:50:36.262 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.690+0000 7f679e9b3700 1 -- 172.21.15.17:0/1889264166 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f679802dc20 con 0x7f6798105840 2024-03-24T02:50:36.262 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.691+0000 7f67957fa700 1 -- 172.21.15.17:0/1889264166 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f6788025480 con 0x7f6798105840 2024-03-24T02:50:36.262 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.691+0000 7f679e9b3700 1 -- 172.21.15.17:0/1889264166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 msgr2=0x7f679819b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:36.262 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.691+0000 7f679e9b3700 1 --2- 172.21.15.17:0/1889264166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 0x7f679819b1d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6788009fb0 tx=0x7f6788004750 comp rx=0 tx=0).stop 2024-03-24T02:50:36.263 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.691+0000 7f679e9b3700 1 -- 172.21.15.17:0/1889264166 shutdown_connections 2024-03-24T02:50:36.263 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.691+0000 7f679e9b3700 1 --2- 172.21.15.17:0/1889264166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6798105840 0x7f679819b1d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:36.263 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.691+0000 7f679e9b3700 1 -- 172.21.15.17:0/1889264166 >> 172.21.15.17:0/1889264166 conn(0x7f6798101300 msgr2=0x7f67981035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:36.263 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.691+0000 7f679e9b3700 1 -- 172.21.15.17:0/1889264166 shutdown_connections 2024-03-24T02:50:36.263 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:35.691+0000 7f679e9b3700 1 -- 172.21.15.17:0/1889264166 wait complete. 2024-03-24T02:50:36.263 INFO:teuthology.orchestra.run.smithi017.stdout:mon is available 2024-03-24T02:50:36.263 INFO:teuthology.orchestra.run.smithi017.stdout:Assimilating anything we can from ceph.conf... 2024-03-24T02:50:37.386 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:37 smithi017 ceph-mon[31551]: from='client.? 172.21.15.17:0/2773153438' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-24T02:50:37.386 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:37 smithi017 ceph-mon[31551]: from='client.? 172.21.15.17:0/2773153438' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-24T02:50:37.395 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:37.395 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [global] 2024-03-24T02:50:37.395 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout fsid = 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:50:37.395 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.17:3300,v1:172.21.15.17:6789] 2024-03-24T02:50:37.395 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-24T02:50:37.395 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-24T02:50:37.395 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-24T02:50:37.395 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [osd] 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.709+0000 7f7918721700 1 Processor -- start 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.709+0000 7f7918721700 1 -- start start 2024-03-24T02:50:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.709+0000 7f7918721700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 0x7f791010a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.709+0000 7f7918721700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791007be50 con 0x7f7910108600 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.709+0000 7f79164bd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 0x7f791010a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.709+0000 7f79164bd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 0x7f791010a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39328/0 (socket says 172.21.15.17:39328) 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.709+0000 7f79164bd700 1 -- 172.21.15.17:0/972322095 learned_addr learned my addr 172.21.15.17:0/972322095 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.709+0000 7f79164bd700 1 -- 172.21.15.17:0/972322095 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791010af30 con 0x7f7910108600 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.710+0000 7f79164bd700 1 --2- 172.21.15.17:0/972322095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 0x7f791010a9f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f790c0098a0 tx=0x7f790c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ad451544726d2d server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.710+0000 7f79154bb700 1 -- 172.21.15.17:0/972322095 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f790c004160 con 0x7f7910108600 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.710+0000 7f79154bb700 1 -- 172.21.15.17:0/972322095 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c004300 con 0x7f7910108600 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.710+0000 7f79154bb700 1 -- 172.21.15.17:0/972322095 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f790c0036a0 con 0x7f7910108600 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.710+0000 7f7918721700 1 -- 172.21.15.17:0/972322095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 msgr2=0x7f791010a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.710+0000 7f7918721700 1 --2- 172.21.15.17:0/972322095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 0x7f791010a9f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f790c0098a0 tx=0x7f790c009bb0 comp rx=0 tx=0).stop 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.710+0000 7f7918721700 1 -- 172.21.15.17:0/972322095 shutdown_connections 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.710+0000 7f7918721700 1 --2- 172.21.15.17:0/972322095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 0x7f791010a9f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:37.397 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.710+0000 7f7918721700 1 -- 172.21.15.17:0/972322095 >> 172.21.15.17:0/972322095 conn(0x7f7910102030 msgr2=0x7f7910104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.710+0000 7f7918721700 1 -- 172.21.15.17:0/972322095 shutdown_connections 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.710+0000 7f7918721700 1 -- 172.21.15.17:0/972322095 wait complete. 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.711+0000 7f7918721700 1 Processor -- start 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.711+0000 7f7918721700 1 -- start start 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.711+0000 7f7918721700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 0x7f791019b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.711+0000 7f7918721700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791010b3a0 con 0x7f7910108600 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.711+0000 7f79164bd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 0x7f791019b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.711+0000 7f79164bd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 0x7f791019b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39342/0 (socket says 172.21.15.17:39342) 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.711+0000 7f79164bd700 1 -- 172.21.15.17:0/2773153438 learned_addr learned my addr 172.21.15.17:0/2773153438 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.712+0000 7f79164bd700 1 -- 172.21.15.17:0/2773153438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791019bab0 con 0x7f7910108600 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.712+0000 7f79164bd700 1 --2- 172.21.15.17:0/2773153438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 0x7f791019b570 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f790c000c00 tx=0x7f790c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.712+0000 7f79077fe700 1 -- 172.21.15.17:0/2773153438 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f790c003db0 con 0x7f7910108600 2024-03-24T02:50:37.398 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.712+0000 7f79077fe700 1 -- 172.21.15.17:0/2773153438 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c011630 con 0x7f7910108600 2024-03-24T02:50:37.399 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.712+0000 7f79077fe700 1 -- 172.21.15.17:0/2773153438 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f790c011950 con 0x7f7910108600 2024-03-24T02:50:37.399 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.712+0000 7f7918721700 1 -- 172.21.15.17:0/2773153438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791019bd40 con 0x7f7910108600 2024-03-24T02:50:37.399 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.712+0000 7f7918721700 1 -- 172.21.15.17:0/2773153438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791019ea30 con 0x7f7910108600 2024-03-24T02:50:37.399 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.712+0000 7f79077fe700 1 -- 172.21.15.17:0/2773153438 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f790c01a460 con 0x7f7910108600 2024-03-24T02:50:37.399 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.712+0000 7f79077fe700 1 -- 172.21.15.17:0/2773153438 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f790c01b780 con 0x7f7910108600 2024-03-24T02:50:37.399 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.713+0000 7f7918721700 1 -- 172.21.15.17:0/2773153438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79100637c0 con 0x7f7910108600 2024-03-24T02:50:37.399 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.715+0000 7f79077fe700 1 -- 172.21.15.17:0/2773153438 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f790c01ba20 con 0x7f7910108600 2024-03-24T02:50:37.399 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.785+0000 7f7918721700 1 -- 172.21.15.17:0/2773153438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f7910193f00 con 0x7f7910108600 2024-03-24T02:50:37.399 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.788+0000 7f79077fe700 1 -- 172.21.15.17:0/2773153438 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f790c01bc00 con 0x7f7910108600 2024-03-24T02:50:37.399 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.789+0000 7f79077fe700 1 -- 172.21.15.17:0/2773153438 <== mon.0 v2:172.21.15.17:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f790c01bde0 con 0x7f7910108600 2024-03-24T02:50:37.399 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.790+0000 7f7918721700 1 -- 172.21.15.17:0/2773153438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 msgr2=0x7f791019b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:37.400 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.790+0000 7f7918721700 1 --2- 172.21.15.17:0/2773153438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 0x7f791019b570 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f790c000c00 tx=0x7f790c004750 comp rx=0 tx=0).stop 2024-03-24T02:50:37.400 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.790+0000 7f7918721700 1 -- 172.21.15.17:0/2773153438 shutdown_connections 2024-03-24T02:50:37.400 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.790+0000 7f7918721700 1 --2- 172.21.15.17:0/2773153438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7910108600 0x7f791019b570 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:37.400 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.790+0000 7f7918721700 1 -- 172.21.15.17:0/2773153438 >> 172.21.15.17:0/2773153438 conn(0x7f7910102030 msgr2=0x7f79101042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:37.400 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.790+0000 7f7918721700 1 -- 172.21.15.17:0/2773153438 shutdown_connections 2024-03-24T02:50:37.400 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:36.790+0000 7f7918721700 1 -- 172.21.15.17:0/2773153438 wait complete. 2024-03-24T02:50:37.401 INFO:teuthology.orchestra.run.smithi017.stdout:Generating new minimal ceph.conf... 2024-03-24T02:50:38.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:38 smithi017 ceph-mon[31551]: from='client.? 172.21.15.17:0/4025580740' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:50:38.699 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.052+0000 7fc488559700 1 Processor -- start 2024-03-24T02:50:38.699 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.052+0000 7fc488559700 1 -- start start 2024-03-24T02:50:38.699 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.053+0000 7fc488559700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 0x7fc480107f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:38.699 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.053+0000 7fc488559700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4801084d0 con 0x7fc480107ae0 2024-03-24T02:50:38.699 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.053+0000 7fc4862f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 0x7fc480107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.053+0000 7fc4862f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 0x7fc480107f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39352/0 (socket says 172.21.15.17:39352) 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.053+0000 7fc4862f5700 1 -- 172.21.15.17:0/596248763 learned_addr learned my addr 172.21.15.17:0/596248763 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.053+0000 7fc4862f5700 1 -- 172.21.15.17:0/596248763 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc480108ce0 con 0x7fc480107ae0 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.054+0000 7fc4862f5700 1 --2- 172.21.15.17:0/596248763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 0x7fc480107f00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0098a0 tx=0x7fc47c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f5339f7a4be6e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.054+0000 7fc4852f3700 1 -- 172.21.15.17:0/596248763 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc47c004160 con 0x7fc480107ae0 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.054+0000 7fc4852f3700 1 -- 172.21.15.17:0/596248763 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc47c004300 con 0x7fc480107ae0 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.054+0000 7fc4852f3700 1 -- 172.21.15.17:0/596248763 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc47c011b50 con 0x7fc480107ae0 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.054+0000 7fc488559700 1 -- 172.21.15.17:0/596248763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 msgr2=0x7fc480107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.054+0000 7fc488559700 1 --2- 172.21.15.17:0/596248763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 0x7fc480107f00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0098a0 tx=0x7fc47c009bb0 comp rx=0 tx=0).stop 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.054+0000 7fc488559700 1 -- 172.21.15.17:0/596248763 shutdown_connections 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.054+0000 7fc488559700 1 --2- 172.21.15.17:0/596248763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 0x7fc480107f00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.054+0000 7fc488559700 1 -- 172.21.15.17:0/596248763 >> 172.21.15.17:0/596248763 conn(0x7fc480103080 msgr2=0x7fc4801054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.054+0000 7fc488559700 1 -- 172.21.15.17:0/596248763 shutdown_connections 2024-03-24T02:50:38.700 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.054+0000 7fc488559700 1 -- 172.21.15.17:0/596248763 wait complete. 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.055+0000 7fc488559700 1 Processor -- start 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.055+0000 7fc488559700 1 -- start start 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.055+0000 7fc488559700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 0x7fc48019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.055+0000 7fc488559700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48010c090 con 0x7fc480107ae0 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.055+0000 7fc4862f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 0x7fc48019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.055+0000 7fc4862f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 0x7fc48019be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39364/0 (socket says 172.21.15.17:39364) 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.055+0000 7fc4862f5700 1 -- 172.21.15.17:0/4025580740 learned_addr learned my addr 172.21.15.17:0/4025580740 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.056+0000 7fc4862f5700 1 -- 172.21.15.17:0/4025580740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48019c340 con 0x7fc480107ae0 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.056+0000 7fc4862f5700 1 --2- 172.21.15.17:0/4025580740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 0x7fc48019be00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009fb0 tx=0x7fc47c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.056+0000 7fc4737fe700 1 -- 172.21.15.17:0/4025580740 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc47c0038b0 con 0x7fc480107ae0 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.056+0000 7fc4737fe700 1 -- 172.21.15.17:0/4025580740 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc47c011e50 con 0x7fc480107ae0 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.056+0000 7fc488559700 1 -- 172.21.15.17:0/4025580740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48019c5d0 con 0x7fc480107ae0 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.056+0000 7fc4737fe700 1 -- 172.21.15.17:0/4025580740 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc47c019550 con 0x7fc480107ae0 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.056+0000 7fc488559700 1 -- 172.21.15.17:0/4025580740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48019f2c0 con 0x7fc480107ae0 2024-03-24T02:50:38.701 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.056+0000 7fc4737fe700 1 -- 172.21.15.17:0/4025580740 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc47c019b90 con 0x7fc480107ae0 2024-03-24T02:50:38.702 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.056+0000 7fc4737fe700 1 -- 172.21.15.17:0/4025580740 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc47c024730 con 0x7fc480107ae0 2024-03-24T02:50:38.702 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.057+0000 7fc488559700 1 -- 172.21.15.17:0/4025580740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4800637c0 con 0x7fc480107ae0 2024-03-24T02:50:38.702 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.059+0000 7fc4737fe700 1 -- 172.21.15.17:0/4025580740 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc47c02dda0 con 0x7fc480107ae0 2024-03-24T02:50:38.702 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.130+0000 7fc488559700 1 -- 172.21.15.17:0/4025580740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fc48002da30 con 0x7fc480107ae0 2024-03-24T02:50:38.702 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.130+0000 7fc4737fe700 1 -- 172.21.15.17:0/4025580740 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fc47c032030 con 0x7fc480107ae0 2024-03-24T02:50:38.702 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.131+0000 7fc488559700 1 -- 172.21.15.17:0/4025580740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 msgr2=0x7fc48019be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:38.702 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.131+0000 7fc488559700 1 --2- 172.21.15.17:0/4025580740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 0x7fc48019be00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009fb0 tx=0x7fc47c011bf0 comp rx=0 tx=0).stop 2024-03-24T02:50:38.702 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.131+0000 7fc488559700 1 -- 172.21.15.17:0/4025580740 shutdown_connections 2024-03-24T02:50:38.702 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.131+0000 7fc488559700 1 --2- 172.21.15.17:0/4025580740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc480107ae0 0x7fc48019be00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:38.702 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.131+0000 7fc488559700 1 -- 172.21.15.17:0/4025580740 >> 172.21.15.17:0/4025580740 conn(0x7fc480103080 msgr2=0x7fc480103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:38.702 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.131+0000 7fc488559700 1 -- 172.21.15.17:0/4025580740 shutdown_connections 2024-03-24T02:50:38.702 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:38.131+0000 7fc488559700 1 -- 172.21.15.17:0/4025580740 wait complete. 2024-03-24T02:50:38.703 INFO:teuthology.orchestra.run.smithi017.stdout:Restarting the monitor... 2024-03-24T02:50:38.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:38 smithi017 systemd[1]: Stopping Ceph mon.a for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:50:39.346 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:38 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a[31547]: 2024-03-24T02:50:38.987+0000 7f6b56373700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T02:50:39.346 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:38 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a[31547]: 2024-03-24T02:50:38.987+0000 7f6b56373700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-24T02:50:39.346 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:39 smithi017 podman[31797]: 2024-03-24 02:50:39.141783709 +0000 UTC m=+0.272861790 container died 575e1c64e2fe5a91fa80b1d3355686bbe43a81fd5c71760ca696c91a6ee7164a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T02:50:39.662 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:39 smithi017 podman[31797]: 2024-03-24 02:50:39.421092024 +0000 UTC m=+0.552170105 container cleanup 575e1c64e2fe5a91fa80b1d3355686bbe43a81fd5c71760ca696c91a6ee7164a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T02:50:39.662 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:39 smithi017 bash[31797]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a 2024-03-24T02:50:39.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:39 smithi017 podman[31812]: 2024-03-24 02:50:39.663671951 +0000 UTC m=+0.522109280 container remove 575e1c64e2fe5a91fa80b1d3355686bbe43a81fd5c71760ca696c91a6ee7164a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-03-24T02:50:39.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:39 smithi017 systemd[1]: ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-24T02:50:39.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:39 smithi017 systemd[1]: Stopped Ceph mon.a for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:50:39.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:39 smithi017 systemd[1]: Starting Ceph mon.a for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:50:40.247 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:39 smithi017 podman[31908]: 2024-03-24 02:50:39.900383347 +0000 UTC m=+0.017658789 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T02:50:40.247 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 podman[31908]: 2024-03-24 02:50:40.014346567 +0000 UTC m=+0.131622058 container create 2644ce8a27f2ce658380e51b2d1b4f55cf03fbc030ad0c47312c0abc4ad41b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-24T02:50:40.248 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 podman[31908]: 2024-03-24 02:50:40.216874318 +0000 UTC m=+0.334149770 container init 2644ce8a27f2ce658380e51b2d1b4f55cf03fbc030ad0c47312c0abc4ad41b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-24T02:50:40.248 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 podman[31908]: 2024-03-24 02:50:40.220290101 +0000 UTC m=+0.337565546 container start 2644ce8a27f2ce658380e51b2d1b4f55cf03fbc030ad0c47312c0abc4ad41b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T02:50:40.248 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T02:50:40.248 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-24T02:50:40.248 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: pidfile_write: ignore empty --pid-file 2024-03-24T02:50:40.248 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: load: jerasure load: lrc 2024-03-24T02:50:40.287 INFO:teuthology.orchestra.run.smithi017.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-24T02:50:40.520 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: RocksDB version: 7.9.2 2024-03-24T02:50:40.520 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Git sha 0 2024-03-24T02:50:40.521 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-24T02:50:40.521 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: DB SUMMARY 2024-03-24T02:50:40.521 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: DB Session ID: SFCN4Z7S8XNS4D8LFG9N 2024-03-24T02:50:40.521 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: CURRENT file: CURRENT 2024-03-24T02:50:40.521 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T02:50:40.521 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-24T02:50:40.521 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-24T02:50:40.521 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100656 ; 2024-03-24T02:50:40.521 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.error_if_exists: 0 2024-03-24T02:50:40.521 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.create_if_missing: 0 2024-03-24T02:50:40.521 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.paranoid_checks: 1 2024-03-24T02:50:40.521 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.env: 0x555bb4626b20 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.info_log: 0x555bb5d8f920 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.statistics: (nil) 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.use_fsync: 0 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_log_file_size: 0 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T02:50:40.522 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.allow_fallocate: 1 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.use_direct_reads: 0 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.db_log_dir: 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.wal_dir: 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T02:50:40.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T02:50:40.524 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T02:50:40.524 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.write_buffer_manager: 0x555bb5d621e0 2024-03-24T02:50:40.524 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T02:50:40.524 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T02:50:40.524 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T02:50:40.524 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T02:50:40.524 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T02:50:40.524 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T02:50:40.524 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T02:50:40.524 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T02:50:40.524 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.unordered_write: 0 2024-03-24T02:50:40.524 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.row_cache: None 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.wal_filter: None 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.two_write_queues: 0 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.wal_compression: 0 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.atomic_flush: 0 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T02:50:40.525 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.log_readahead_size: 0 2024-03-24T02:50:40.526 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T02:50:40.526 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T02:50:40.526 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T02:50:40.526 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T02:50:40.526 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T02:50:40.526 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T02:50:40.526 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T02:50:40.526 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_background_jobs: 2 2024-03-24T02:50:40.526 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_background_compactions: -1 2024-03-24T02:50:40.526 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_subcompactions: 1 2024-03-24T02:50:40.527 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T02:50:40.527 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_open_files: -1 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_background_flushes: -1 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Compression algorithms supported: 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T02:50:40.528 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: kZSTD supported: 0 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: kXpressCompression supported: 0 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: kZlibCompression supported: 1 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: kSnappyCompression supported: 1 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: kLZ4Compression supported: 1 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: kBZip2Compression supported: 0 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.merge_operator: 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_filter: None 2024-03-24T02:50:40.529 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555bb5d8fa20) 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout: cache_index_and_filter_blocks: 1 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout: pin_top_level_index_and_filter: 1 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout: index_type: 0 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout: data_block_index_type: 0 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout: index_shortening: 1 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T02:50:40.530 INFO:journalctl@ceph.mon.a.smithi017.stdout: checksum: 4 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: no_block_cache: 0 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_cache: 0x555bb5d95090 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_cache_name: BinnedLRUCache 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_cache_options: 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: capacity : 536870912 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: num_shard_bits : 4 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: strict_capacity_limit : 0 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: high_pri_pool_ratio: 0.000 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_cache_compressed: (nil) 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: persistent_cache: (nil) 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_size: 4096 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_size_deviation: 10 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_restart_interval: 16 2024-03-24T02:50:40.531 INFO:journalctl@ceph.mon.a.smithi017.stdout: index_block_restart_interval: 1 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: metadata_block_size: 4096 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: partition_filters: 0 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: use_delta_encoding: 1 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: filter_policy: bloomfilter 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: whole_key_filtering: 1 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: verify_compression: 0 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: read_amp_bytes_per_bit: 0 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: format_version: 5 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: enable_index_compression: 1 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_align: 0 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: max_auto_readahead_size: 262144 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: prepopulate_block_cache: 0 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: initial_auto_readahead_size: 8192 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T02:50:40.532 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compression: NoCompression 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.num_levels: 7 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T02:50:40.533 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T02:50:40.534 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T02:50:40.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T02:50:40.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.table_properties_collectors: 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.inplace_update_support: 0 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.bloom_locality: 0 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.max_successive_merges: 0 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.ttl: 2592000 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T02:50:40.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.enable_blob_files: false 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.min_blob_size: 0 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-24T02:50:40.538 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a0806e21-5d24-4c74-b53e-d135e42c8b93 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711248640250935, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711248640251961, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95790, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93780, "index_size": 292, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13914, "raw_average_key_size": 50, "raw_value_size": 86213, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711248640, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a0806e21-5d24-4c74-b53e-d135e42c8b93", "db_session_id": "SFCN4Z7S8XNS4D8LFG9N", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711248640252045, "job": 1, "event": "recovery_finished"} 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555bb5e88000 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: DB pointer 0x555bb5e72000 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout: ** DB Stats ** 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T02:50:40.539 INFO:journalctl@ceph.mon.a.smithi017.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: ** Compaction Stats [default] ** 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: L0 2/0 95.38 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: Sum 2/0 95.38 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: ** Compaction Stats [default] ** 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T02:50:40.540 INFO:journalctl@ceph.mon.a.smithi017.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 135.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: Cumulative compaction: 0.00 GB write, 29.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: Interval compaction: 0.00 GB write, 29.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: Block cache BinnedLRUCache@0x555bb5d95090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-24T02:50:40.541 INFO:journalctl@ceph.mon.a.smithi017.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: starting mon.a rank 0 at public addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] at bind addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: mon.a@-1(???) e1 preinit fsid 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: mon.a@-1(???).mds e1 new map 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: mon.a@-1(???).mds e1 print_map 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout: e1 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout: legacy client fscid: -1 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout: No filesystems configured 2024-03-24T02:50:40.542 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-24T02:50:40.543 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T02:50:40.543 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T02:50:40.543 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T02:50:40.543 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 ceph-mon[31926]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-24T02:50:40.543 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 bash[31908]: 2644ce8a27f2ce658380e51b2d1b4f55cf03fbc030ad0c47312c0abc4ad41b4b 2024-03-24T02:50:40.543 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:40 smithi017 systemd[1]: Started Ceph mon.a for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:50:41.379 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:41 smithi017 ceph-mon[31926]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-24T02:50:41.380 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:41 smithi017 ceph-mon[31926]: monmap e1: 1 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T02:50:41.380 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:41 smithi017 ceph-mon[31926]: fsmap 2024-03-24T02:50:41.380 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:41 smithi017 ceph-mon[31926]: osdmap e1: 0 total, 0 up, 0 in 2024-03-24T02:50:41.380 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:41 smithi017 ceph-mon[31926]: mgrmap e1: no daemons active 2024-03-24T02:50:41.380 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:41 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1286218398' entity='client.admin' 2024-03-24T02:50:41.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.722+0000 7f9a5fb1c700 1 Processor -- start 2024-03-24T02:50:41.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.723+0000 7f9a5fb1c700 1 -- start start 2024-03-24T02:50:41.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.723+0000 7f9a5fb1c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 0x7f9a58107f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:41.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.723+0000 7f9a5fb1c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a581084f0 con 0x7f9a58107b00 2024-03-24T02:50:41.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.723+0000 7f9a5d8b8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 0x7f9a58107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:41.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.723+0000 7f9a5d8b8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 0x7f9a58107f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60532/0 (socket says 172.21.15.17:60532) 2024-03-24T02:50:41.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.723+0000 7f9a5d8b8700 1 -- 172.21.15.17:0/550506205 learned_addr learned my addr 172.21.15.17:0/550506205 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:41.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.723+0000 7f9a5d8b8700 1 -- 172.21.15.17:0/550506205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a58108d00 con 0x7f9a58107b00 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.724+0000 7f9a5d8b8700 1 --2- 172.21.15.17:0/550506205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 0x7f9a58107f20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9a540098a0 tx=0x7f9a54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcce7cd63d95aef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.724+0000 7f9a5c8b6700 1 -- 172.21.15.17:0/550506205 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a54004160 con 0x7f9a58107b00 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.724+0000 7f9a5c8b6700 1 -- 172.21.15.17:0/550506205 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a54004300 con 0x7f9a58107b00 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.724+0000 7f9a5c8b6700 1 -- 172.21.15.17:0/550506205 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a54011b50 con 0x7f9a58107b00 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.724+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/550506205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 msgr2=0x7f9a58107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.724+0000 7f9a5fb1c700 1 --2- 172.21.15.17:0/550506205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 0x7f9a58107f20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9a540098a0 tx=0x7f9a54009bb0 comp rx=0 tx=0).stop 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.724+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/550506205 shutdown_connections 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.724+0000 7f9a5fb1c700 1 --2- 172.21.15.17:0/550506205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 0x7f9a58107f20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.724+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/550506205 >> 172.21.15.17:0/550506205 conn(0x7f9a58103120 msgr2=0x7f9a58105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.724+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/550506205 shutdown_connections 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.724+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/550506205 wait complete. 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.725+0000 7f9a5fb1c700 1 Processor -- start 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.725+0000 7f9a5fb1c700 1 -- start start 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.725+0000 7f9a5fb1c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 0x7f9a5819be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:41.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.725+0000 7f9a5fb1c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5810c0b0 con 0x7f9a58107b00 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.725+0000 7f9a5d8b8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 0x7f9a5819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.725+0000 7f9a5d8b8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 0x7f9a5819be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60546/0 (socket says 172.21.15.17:60546) 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.725+0000 7f9a5d8b8700 1 -- 172.21.15.17:0/1286218398 learned_addr learned my addr 172.21.15.17:0/1286218398 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.726+0000 7f9a5d8b8700 1 -- 172.21.15.17:0/1286218398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5819c360 con 0x7f9a58107b00 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.726+0000 7f9a5d8b8700 1 --2- 172.21.15.17:0/1286218398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 0x7f9a5819be20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9a54009fb0 tx=0x7f9a54011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.726+0000 7f9a4effd700 1 -- 172.21.15.17:0/1286218398 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a540038b0 con 0x7f9a58107b00 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.726+0000 7f9a4effd700 1 -- 172.21.15.17:0/1286218398 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a54011e50 con 0x7f9a58107b00 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.726+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/1286218398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5819c5f0 con 0x7f9a58107b00 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.726+0000 7f9a4effd700 1 -- 172.21.15.17:0/1286218398 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a54019550 con 0x7f9a58107b00 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.726+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/1286218398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5819f2e0 con 0x7f9a58107b00 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.726+0000 7f9a4effd700 1 -- 172.21.15.17:0/1286218398 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9a54019b90 con 0x7f9a58107b00 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.726+0000 7f9a4effd700 1 -- 172.21.15.17:0/1286218398 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9a54024900 con 0x7f9a58107b00 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.727+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/1286218398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a580637c0 con 0x7f9a58107b00 2024-03-24T02:50:41.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.729+0000 7f9a4effd700 1 -- 172.21.15.17:0/1286218398 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9a54003a50 con 0x7f9a58107b00 2024-03-24T02:50:41.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.799+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/1286218398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f9a58075a10 con 0x7f9a58107b00 2024-03-24T02:50:41.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.801+0000 7f9a4effd700 1 -- 172.21.15.17:0/1286218398 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9a54032030 con 0x7f9a58107b00 2024-03-24T02:50:41.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.802+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/1286218398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 msgr2=0x7f9a5819be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:41.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.802+0000 7f9a5fb1c700 1 --2- 172.21.15.17:0/1286218398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 0x7f9a5819be20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9a54009fb0 tx=0x7f9a54011bf0 comp rx=0 tx=0).stop 2024-03-24T02:50:41.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.802+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/1286218398 shutdown_connections 2024-03-24T02:50:41.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.802+0000 7f9a5fb1c700 1 --2- 172.21.15.17:0/1286218398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a58107b00 0x7f9a5819be20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:41.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.802+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/1286218398 >> 172.21.15.17:0/1286218398 conn(0x7f9a58103120 msgr2=0x7f9a58103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:41.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.802+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/1286218398 shutdown_connections 2024-03-24T02:50:41.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:40.802+0000 7f9a5fb1c700 1 -- 172.21.15.17:0/1286218398 wait complete. 2024-03-24T02:50:41.389 INFO:teuthology.orchestra.run.smithi017.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-24T02:50:41.413 INFO:teuthology.orchestra.run.smithi017.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-24T02:50:41.413 INFO:teuthology.orchestra.run.smithi017.stdout:Creating mgr... 2024-03-24T02:50:41.414 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-24T02:50:41.414 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-24T02:50:41.679 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.x 2024-03-24T02:50:41.679 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.x.service: Unit ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.x.service not loaded. 2024-03-24T02:50:41.828 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2a682800-e989-11ee-95cd-87774f69a715.target.wants/ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-2a682800-e989-11ee-95cd-87774f69a715@.service. 2024-03-24T02:50:41.956 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:41 smithi017 systemd[1]: Starting Ceph mgr.x for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:50:42.334 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:42 smithi017 podman[32146]: 2024-03-24 02:50:42.058384225 +0000 UTC m=+0.113453339 container create 13a48e6ed072e3aca460a802d9718e5995dd96f48915160717126c56405e342b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-24T02:50:42.334 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:42 smithi017 podman[32146]: 2024-03-24 02:50:41.962977617 +0000 UTC m=+0.018046711 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T02:50:42.334 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:42 smithi017 podman[32146]: 2024-03-24 02:50:42.260422451 +0000 UTC m=+0.315491546 container init 13a48e6ed072e3aca460a802d9718e5995dd96f48915160717126c56405e342b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T02:50:42.334 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:42 smithi017 podman[32146]: 2024-03-24 02:50:42.262709919 +0000 UTC m=+0.317778995 container start 13a48e6ed072e3aca460a802d9718e5995dd96f48915160717126c56405e342b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-24T02:50:42.334 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:42 smithi017 bash[32146]: 13a48e6ed072e3aca460a802d9718e5995dd96f48915160717126c56405e342b 2024-03-24T02:50:42.347 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T02:50:42.347 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout disabled 2024-03-24T02:50:42.353 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T02:50:42.353 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2024-03-24T02:50:42.354 INFO:teuthology.orchestra.run.smithi017.stdout:firewalld.service is not enabled 2024-03-24T02:50:42.354 INFO:teuthology.orchestra.run.smithi017.stdout:Not possible to enable service . firewalld.service is not available 2024-03-24T02:50:42.362 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T02:50:42.362 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout disabled 2024-03-24T02:50:42.369 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T02:50:42.370 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2024-03-24T02:50:42.370 INFO:teuthology.orchestra.run.smithi017.stdout:firewalld.service is not enabled 2024-03-24T02:50:42.370 INFO:teuthology.orchestra.run.smithi017.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-24T02:50:42.370 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mgr to start... 2024-03-24T02:50:42.370 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mgr... 2024-03-24T02:50:42.590 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:42 smithi017 systemd[1]: Started Ceph mgr.x for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:50:42.854 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:42 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:42.824+0000 7f62ebdb2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T02:50:43.157 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:42 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1635072261' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T02:50:43.157 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:42 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:42.925+0000 7f62ebdb2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T02:50:43.445 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 0 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T02:50:43.446 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "a" 2024-03-24T02:50:43.447 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-03-24T02:50:43.447 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-24T02:50:43.447 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T02:50:43.447 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:43.447 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-24T02:50:43.447 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T02:50:43.447 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:43.447 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T02:50:43.447 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:43.447 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T02:50:43.447 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T02:50:43.447 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T02:50:43.455 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T02:50:43.455 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T02:50:43.455 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T02:50:43.455 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:43.455 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T02:50:43.455 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T02:50:43.455 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T02:50:43.455 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T02:50:43.455 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T02:50:43.456 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T02:50:35.153134+0000", 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.771+0000 7f3570f8a700 1 Processor -- start 2024-03-24T02:50:43.457 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.771+0000 7f3570f8a700 1 -- start start 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.771+0000 7f3570f8a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 0x7f356c10aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.771+0000 7f3570f8a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c07bdf0 con 0x7f356c108620 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.771+0000 7f356a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 0x7f356c10aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.771+0000 7f356a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 0x7f356c10aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60574/0 (socket says 172.21.15.17:60574) 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.771+0000 7f356a59c700 1 -- 172.21.15.17:0/2388136722 learned_addr learned my addr 172.21.15.17:0/2388136722 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.772+0000 7f356a59c700 1 -- 172.21.15.17:0/2388136722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c10af50 con 0x7f356c108620 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.772+0000 7f356a59c700 1 --2- 172.21.15.17:0/2388136722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 0x7f356c10aa10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f355c0098a0 tx=0x7f355c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73297efb8633904 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.772+0000 7f356959a700 1 -- 172.21.15.17:0/2388136722 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f355c004160 con 0x7f356c108620 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.772+0000 7f356959a700 1 -- 172.21.15.17:0/2388136722 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f355c004300 con 0x7f356c108620 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.772+0000 7f356959a700 1 -- 172.21.15.17:0/2388136722 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f355c011b50 con 0x7f356c108620 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.773+0000 7f3570f8a700 1 -- 172.21.15.17:0/2388136722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 msgr2=0x7f356c10aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.773+0000 7f3570f8a700 1 --2- 172.21.15.17:0/2388136722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 0x7f356c10aa10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f355c0098a0 tx=0x7f355c009bb0 comp rx=0 tx=0).stop 2024-03-24T02:50:43.458 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.773+0000 7f3570f8a700 1 -- 172.21.15.17:0/2388136722 shutdown_connections 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.773+0000 7f3570f8a700 1 --2- 172.21.15.17:0/2388136722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 0x7f356c10aa10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.773+0000 7f3570f8a700 1 -- 172.21.15.17:0/2388136722 >> 172.21.15.17:0/2388136722 conn(0x7f356c101fb0 msgr2=0x7f356c1043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.773+0000 7f3570f8a700 1 -- 172.21.15.17:0/2388136722 shutdown_connections 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.773+0000 7f3570f8a700 1 -- 172.21.15.17:0/2388136722 wait complete. 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.773+0000 7f3570f8a700 1 Processor -- start 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f3570f8a700 1 -- start start 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f3570f8a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 0x7f356c107ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f3570f8a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c10bf10 con 0x7f356c108620 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f356a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 0x7f356c107ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f356a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 0x7f356c107ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60584/0 (socket says 172.21.15.17:60584) 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f356a59c700 1 -- 172.21.15.17:0/1635072261 learned_addr learned my addr 172.21.15.17:0/1635072261 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f356a59c700 1 -- 172.21.15.17:0/1635072261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c1083e0 con 0x7f356c108620 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f356a59c700 1 --2- 172.21.15.17:0/1635072261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 0x7f356c107ea0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f355c009fb0 tx=0x7f355c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f355b7fe700 1 -- 172.21.15.17:0/1635072261 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f355c0038b0 con 0x7f356c108620 2024-03-24T02:50:43.459 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f355b7fe700 1 -- 172.21.15.17:0/1635072261 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f355c011e50 con 0x7f356c108620 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f355b7fe700 1 -- 172.21.15.17:0/1635072261 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f355c019420 con 0x7f356c108620 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f3570f8a700 1 -- 172.21.15.17:0/1635072261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356c1065d0 con 0x7f356c108620 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.774+0000 7f3570f8a700 1 -- 172.21.15.17:0/1635072261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356c106b10 con 0x7f356c108620 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.775+0000 7f355b7fe700 1 -- 172.21.15.17:0/1635072261 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f355c019ab0 con 0x7f356c108620 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.775+0000 7f355b7fe700 1 -- 172.21.15.17:0/1635072261 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f355c024730 con 0x7f356c108620 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.776+0000 7f3570f8a700 1 -- 172.21.15.17:0/1635072261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356c0637c0 con 0x7f356c108620 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.780+0000 7f355b7fe700 1 -- 172.21.15.17:0/1635072261 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f355c043dd0 con 0x7f356c108620 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.853+0000 7f3570f8a700 1 -- 172.21.15.17:0/1635072261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f356c195440 con 0x7f356c108620 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.854+0000 7f355b7fe700 1 -- 172.21.15.17:0/1635072261 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f355c032030 con 0x7f356c108620 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.855+0000 7f3570f8a700 1 -- 172.21.15.17:0/1635072261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 msgr2=0x7f356c107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.855+0000 7f3570f8a700 1 --2- 172.21.15.17:0/1635072261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 0x7f356c107ea0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f355c009fb0 tx=0x7f355c011bf0 comp rx=0 tx=0).stop 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.855+0000 7f3570f8a700 1 -- 172.21.15.17:0/1635072261 shutdown_connections 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.855+0000 7f3570f8a700 1 --2- 172.21.15.17:0/1635072261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f356c108620 0x7f356c107ea0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:43.460 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.855+0000 7f3570f8a700 1 -- 172.21.15.17:0/1635072261 >> 172.21.15.17:0/1635072261 conn(0x7f356c101fb0 msgr2=0x7f356c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:43.461 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.855+0000 7f3570f8a700 1 -- 172.21.15.17:0/1635072261 shutdown_connections 2024-03-24T02:50:43.461 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:42.855+0000 7f3570f8a700 1 -- 172.21.15.17:0/1635072261 wait complete. 2024-03-24T02:50:43.461 INFO:teuthology.orchestra.run.smithi017.stdout:mgr not available, waiting (1/15)... 2024-03-24T02:50:43.705 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:43 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:43.589+0000 7f62ebdb2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T02:50:43.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:43 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:43.683+0000 7f62ebdb2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T02:50:44.206 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:43 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:43.825+0000 7f62ebdb2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T02:50:44.206 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:43 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:43.922+0000 7f62ebdb2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T02:50:44.206 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:44 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:44.006+0000 7f62ebdb2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T02:50:44.567 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:44 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:44.273+0000 7f62ebdb2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T02:50:44.567 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:44 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:44.384+0000 7f62ebdb2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T02:50:44.567 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:44 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:44.471+0000 7f62ebdb2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T02:50:44.567 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:44 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:44.567+0000 7f62ebdb2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T02:50:44.955 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:44 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:44.744+0000 7f62ebdb2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T02:50:45.843 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:45 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:45.764+0000 7f62ebdb2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T02:50:46.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:45 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/636798484' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T02:50:46.206 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:45 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:45.902+0000 7f62ebdb2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T02:50:46.546 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:46 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:46.254+0000 7f62ebdb2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T02:50:46.546 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:46 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:46.346+0000 7f62ebdb2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T02:50:46.546 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:46 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:46.546+0000 7f62ebdb2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T02:50:46.583 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:46.583 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-03-24T02:50:46.583 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T02:50:46.583 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 0 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "a" 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:46.584 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T02:50:46.585 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T02:50:46.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T02:50:35.153134+0000", 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.860+0000 7f425f626700 1 Processor -- start 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.860+0000 7f425f626700 1 -- start start 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.860+0000 7f425f626700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 0x7f425810a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:46.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.860+0000 7f425f626700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425807be30 con 0x7f42581085b0 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.860+0000 7f425d3c2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 0x7f425810a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.860+0000 7f425d3c2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 0x7f425810a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60598/0 (socket says 172.21.15.17:60598) 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.860+0000 7f425d3c2700 1 -- 172.21.15.17:0/11472815 learned_addr learned my addr 172.21.15.17:0/11472815 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.861+0000 7f425d3c2700 1 -- 172.21.15.17:0/11472815 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425810aee0 con 0x7f42581085b0 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.861+0000 7f425d3c2700 1 --2- 172.21.15.17:0/11472815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 0x7f425810a9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42540098a0 tx=0x7f4254009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91a5aa3de03e0a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.861+0000 7f424ffff700 1 -- 172.21.15.17:0/11472815 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4254004160 con 0x7f42581085b0 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.861+0000 7f424ffff700 1 -- 172.21.15.17:0/11472815 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4254004300 con 0x7f42581085b0 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.861+0000 7f424ffff700 1 -- 172.21.15.17:0/11472815 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4254011b50 con 0x7f42581085b0 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.862+0000 7f425f626700 1 -- 172.21.15.17:0/11472815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 msgr2=0x7f425810a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.862+0000 7f425f626700 1 --2- 172.21.15.17:0/11472815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 0x7f425810a9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42540098a0 tx=0x7f4254009bb0 comp rx=0 tx=0).stop 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.862+0000 7f425f626700 1 -- 172.21.15.17:0/11472815 shutdown_connections 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.862+0000 7f425f626700 1 --2- 172.21.15.17:0/11472815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 0x7f425810a9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.862+0000 7f425f626700 1 -- 172.21.15.17:0/11472815 >> 172.21.15.17:0/11472815 conn(0x7f4258102000 msgr2=0x7f4258104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.862+0000 7f425f626700 1 -- 172.21.15.17:0/11472815 shutdown_connections 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.862+0000 7f425f626700 1 -- 172.21.15.17:0/11472815 wait complete. 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.862+0000 7f425f626700 1 Processor -- start 2024-03-24T02:50:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.862+0000 7f425f626700 1 -- start start 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.862+0000 7f425f626700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 0x7f4258107e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.862+0000 7f425f626700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425810beb0 con 0x7f42581085b0 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.863+0000 7f425d3c2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 0x7f4258107e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.863+0000 7f425d3c2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 0x7f4258107e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60610/0 (socket says 172.21.15.17:60610) 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.863+0000 7f425d3c2700 1 -- 172.21.15.17:0/636798484 learned_addr learned my addr 172.21.15.17:0/636798484 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.863+0000 7f425d3c2700 1 -- 172.21.15.17:0/636798484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4258108370 con 0x7f42581085b0 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.863+0000 7f425d3c2700 1 --2- 172.21.15.17:0/636798484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 0x7f4258107e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4254009fb0 tx=0x7f4254011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.863+0000 7f424e7fc700 1 -- 172.21.15.17:0/636798484 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42540038b0 con 0x7f42581085b0 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.863+0000 7f424e7fc700 1 -- 172.21.15.17:0/636798484 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4254011e50 con 0x7f42581085b0 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.863+0000 7f425f626700 1 -- 172.21.15.17:0/636798484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4258106560 con 0x7f42581085b0 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.863+0000 7f424e7fc700 1 -- 172.21.15.17:0/636798484 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4254019420 con 0x7f42581085b0 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.863+0000 7f425f626700 1 -- 172.21.15.17:0/636798484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4258106aa0 con 0x7f42581085b0 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.864+0000 7f424e7fc700 1 -- 172.21.15.17:0/636798484 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4254019a80 con 0x7f42581085b0 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.864+0000 7f424e7fc700 1 -- 172.21.15.17:0/636798484 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4254024930 con 0x7f42581085b0 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.864+0000 7f425f626700 1 -- 172.21.15.17:0/636798484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42580637c0 con 0x7f42581085b0 2024-03-24T02:50:46.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.866+0000 7f424e7fc700 1 -- 172.21.15.17:0/636798484 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4254043dd0 con 0x7f42581085b0 2024-03-24T02:50:46.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.941+0000 7f425f626700 1 -- 172.21.15.17:0/636798484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4258106d50 con 0x7f42581085b0 2024-03-24T02:50:46.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.941+0000 7f424e7fc700 1 -- 172.21.15.17:0/636798484 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f4254032020 con 0x7f42581085b0 2024-03-24T02:50:46.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.942+0000 7f425f626700 1 -- 172.21.15.17:0/636798484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 msgr2=0x7f4258107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:46.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.942+0000 7f425f626700 1 --2- 172.21.15.17:0/636798484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 0x7f4258107e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4254009fb0 tx=0x7f4254011bf0 comp rx=0 tx=0).stop 2024-03-24T02:50:46.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.942+0000 7f425f626700 1 -- 172.21.15.17:0/636798484 shutdown_connections 2024-03-24T02:50:46.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.942+0000 7f425f626700 1 --2- 172.21.15.17:0/636798484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42581085b0 0x7f4258107e30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:46.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.942+0000 7f425f626700 1 -- 172.21.15.17:0/636798484 >> 172.21.15.17:0/636798484 conn(0x7f4258102000 msgr2=0x7f4258102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:46.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.943+0000 7f425f626700 1 -- 172.21.15.17:0/636798484 shutdown_connections 2024-03-24T02:50:46.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:45.943+0000 7f425f626700 1 -- 172.21.15.17:0/636798484 wait complete. 2024-03-24T02:50:46.590 INFO:teuthology.orchestra.run.smithi017.stdout:mgr not available, waiting (2/15)... 2024-03-24T02:50:46.955 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:46 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:46.738+0000 7f62ebdb2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T02:50:47.290 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:47 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:47.030+0000 7f62ebdb2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T02:50:47.290 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:47 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:47.290+0000 7f62ebdb2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T02:50:47.705 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:47 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:47.533+0000 7f62ebdb2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T02:50:48.704 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:48 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:48.358+0000 7f62ebdb2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T02:50:48.955 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:48 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:48.947+0000 7f62ebdb2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T02:50:49.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:49 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4294827309' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T02:50:49.456 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:49 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:49.052+0000 7f62ebdb2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T02:50:49.456 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:49 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:49.279+0000 7f62ebdb2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T02:50:49.750 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:49.750 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-03-24T02:50:49.750 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T02:50:49.750 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T02:50:49.750 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T02:50:49.750 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T02:50:49.750 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 0 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "a" 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-24T02:50:49.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T02:50:49.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:49.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T02:50:49.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:49.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T02:50:49.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T02:50:49.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T02:50:49.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T02:50:49.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T02:50:49.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T02:50:49.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:49.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T02:50:49.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T02:50:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T02:50:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T02:50:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T02:50:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T02:50:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T02:50:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T02:50:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T02:50:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T02:50:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T02:50:49.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T02:50:49.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T02:50:35.153134+0000", 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.039+0000 7f4531505700 1 Processor -- start 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.039+0000 7f4531505700 1 -- start start 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.040+0000 7f4531505700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 0x7f452c10a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.040+0000 7f4531505700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c07be90 con 0x7f452c1085c0 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.040+0000 7f452affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 0x7f452c10a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.040+0000 7f452affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 0x7f452c10a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60616/0 (socket says 172.21.15.17:60616) 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.040+0000 7f452affd700 1 -- 172.21.15.17:0/3611648775 learned_addr learned my addr 172.21.15.17:0/3611648775 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:49.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.040+0000 7f452affd700 1 -- 172.21.15.17:0/3611648775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c10aef0 con 0x7f452c1085c0 2024-03-24T02:50:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.040+0000 7f452affd700 1 --2- 172.21.15.17:0/3611648775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 0x7f452c10a9b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098a0 tx=0x7f451c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d90e3135015502d server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.040+0000 7f4529ffb700 1 -- 172.21.15.17:0/3611648775 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f451c004160 con 0x7f452c1085c0 2024-03-24T02:50:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.040+0000 7f4529ffb700 1 -- 172.21.15.17:0/3611648775 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f451c004300 con 0x7f452c1085c0 2024-03-24T02:50:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.040+0000 7f4529ffb700 1 -- 172.21.15.17:0/3611648775 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f451c011b50 con 0x7f452c1085c0 2024-03-24T02:50:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.041+0000 7f4531505700 1 -- 172.21.15.17:0/3611648775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 msgr2=0x7f452c10a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.041+0000 7f4531505700 1 --2- 172.21.15.17:0/3611648775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 0x7f452c10a9b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098a0 tx=0x7f451c009bb0 comp rx=0 tx=0).stop 2024-03-24T02:50:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.041+0000 7f4531505700 1 -- 172.21.15.17:0/3611648775 shutdown_connections 2024-03-24T02:50:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.041+0000 7f4531505700 1 --2- 172.21.15.17:0/3611648775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 0x7f452c10a9b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.041+0000 7f4531505700 1 -- 172.21.15.17:0/3611648775 >> 172.21.15.17:0/3611648775 conn(0x7f452c102030 msgr2=0x7f452c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.041+0000 7f4531505700 1 -- 172.21.15.17:0/3611648775 shutdown_connections 2024-03-24T02:50:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.041+0000 7f4531505700 1 -- 172.21.15.17:0/3611648775 wait complete. 2024-03-24T02:50:49.756 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.042+0000 7f4531505700 1 Processor -- start 2024-03-24T02:50:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.042+0000 7f4531505700 1 -- start start 2024-03-24T02:50:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.042+0000 7f4531505700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 0x7f452c19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.042+0000 7f4531505700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c10bf20 con 0x7f452c1085c0 2024-03-24T02:50:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.042+0000 7f452affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 0x7f452c19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.042+0000 7f452affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 0x7f452c19be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60624/0 (socket says 172.21.15.17:60624) 2024-03-24T02:50:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.042+0000 7f452affd700 1 -- 172.21.15.17:0/4294827309 learned_addr learned my addr 172.21.15.17:0/4294827309 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.042+0000 7f452affd700 1 -- 172.21.15.17:0/4294827309 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c19c370 con 0x7f452c1085c0 2024-03-24T02:50:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.043+0000 7f452affd700 1 --2- 172.21.15.17:0/4294827309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 0x7f452c19be30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f451c009fb0 tx=0x7f451c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.043+0000 7f4513fff700 1 -- 172.21.15.17:0/4294827309 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f451c0038b0 con 0x7f452c1085c0 2024-03-24T02:50:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.043+0000 7f4531505700 1 -- 172.21.15.17:0/4294827309 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c19c600 con 0x7f452c1085c0 2024-03-24T02:50:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.043+0000 7f4513fff700 1 -- 172.21.15.17:0/4294827309 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f451c011e50 con 0x7f452c1085c0 2024-03-24T02:50:49.757 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.043+0000 7f4513fff700 1 -- 172.21.15.17:0/4294827309 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f451c019420 con 0x7f452c1085c0 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.043+0000 7f4531505700 1 -- 172.21.15.17:0/4294827309 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c19f2f0 con 0x7f452c1085c0 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.043+0000 7f4513fff700 1 -- 172.21.15.17:0/4294827309 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f451c019ad0 con 0x7f452c1085c0 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.043+0000 7f4513fff700 1 -- 172.21.15.17:0/4294827309 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f451c024730 con 0x7f452c1085c0 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.044+0000 7f4531505700 1 -- 172.21.15.17:0/4294827309 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c0637c0 con 0x7f452c1085c0 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.048+0000 7f4513fff700 1 -- 172.21.15.17:0/4294827309 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f451c043dd0 con 0x7f452c1085c0 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.121+0000 7f4531505700 1 -- 172.21.15.17:0/4294827309 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f452c02da30 con 0x7f452c1085c0 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.121+0000 7f4513fff700 1 -- 172.21.15.17:0/4294827309 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f451c032030 con 0x7f452c1085c0 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.122+0000 7f4531505700 1 -- 172.21.15.17:0/4294827309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 msgr2=0x7f452c19be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.122+0000 7f4531505700 1 --2- 172.21.15.17:0/4294827309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 0x7f452c19be30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f451c009fb0 tx=0x7f451c011bf0 comp rx=0 tx=0).stop 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.122+0000 7f4531505700 1 -- 172.21.15.17:0/4294827309 shutdown_connections 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.122+0000 7f4531505700 1 --2- 172.21.15.17:0/4294827309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f452c1085c0 0x7f452c19be30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.122+0000 7f4531505700 1 -- 172.21.15.17:0/4294827309 >> 172.21.15.17:0/4294827309 conn(0x7f452c102030 msgr2=0x7f452c102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.123+0000 7f4531505700 1 -- 172.21.15.17:0/4294827309 shutdown_connections 2024-03-24T02:50:49.758 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:49.123+0000 7f4531505700 1 -- 172.21.15.17:0/4294827309 wait complete. 2024-03-24T02:50:49.759 INFO:teuthology.orchestra.run.smithi017.stdout:mgr not available, waiting (3/15)... 2024-03-24T02:50:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: Activating manager daemon x 2024-03-24T02:50:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: mgrmap e2: x(active, starting, since 0.00298857s) 2024-03-24T02:50:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: from='mgr.14100 172.21.15.17:0/1298079487' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T02:50:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: from='mgr.14100 172.21.15.17:0/1298079487' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T02:50:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: from='mgr.14100 172.21.15.17:0/1298079487' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T02:50:50.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: from='mgr.14100 172.21.15.17:0/1298079487' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:50:50.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: from='mgr.14100 172.21.15.17:0/1298079487' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T02:50:50.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: Manager daemon x is now available 2024-03-24T02:50:50.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: from='mgr.14100 172.21.15.17:0/1298079487' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-24T02:50:50.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: from='mgr.14100 172.21.15.17:0/1298079487' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-24T02:50:50.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: from='mgr.14100 172.21.15.17:0/1298079487' entity='mgr.x' 2024-03-24T02:50:50.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: from='mgr.14100 172.21.15.17:0/1298079487' entity='mgr.x' 2024-03-24T02:50:50.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:50 smithi017 ceph-mon[31926]: from='mgr.14100 172.21.15.17:0/1298079487' entity='mgr.x' 2024-03-24T02:50:51.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:51 smithi017 ceph-mon[31926]: mgrmap e3: x(active, since 1.00762s) 2024-03-24T02:50:53.081 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:53.081 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-03-24T02:50:53.081 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T02:50:53.081 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T02:50:53.081 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T02:50:53.081 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T02:50:53.081 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 0 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "a" 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-24T02:50:53.082 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T02:50:53.083 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:53.083 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T02:50:53.083 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:53.083 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T02:50:53.083 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T02:50:53.083 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T02:50:53.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T02:50:53.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-03-24T02:50:53.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T02:50:53.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:53.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T02:50:53.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T02:50:53.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T02:50:35.153134+0000", 2024-03-24T02:50:53.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T02:50:53.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-03-24T02:50:53.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T02:50:53.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-03-24T02:50:53.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.185+0000 7f1d7e962700 1 Processor -- start 2024-03-24T02:50:53.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.185+0000 7f1d7e962700 1 -- start start 2024-03-24T02:50:53.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.186+0000 7f1d7e962700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 0x7f1d7807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.186+0000 7f1d7e962700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7807c8f0 con 0x7f1d7807dec0 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.186+0000 7f1d77fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 0x7f1d7807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.186+0000 7f1d77fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 0x7f1d7807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38364/0 (socket says 172.21.15.17:38364) 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.186+0000 7f1d77fff700 1 -- 172.21.15.17:0/1996514383 learned_addr learned my addr 172.21.15.17:0/1996514383 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.186+0000 7f1d77fff700 1 -- 172.21.15.17:0/1996514383 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7807ca70 con 0x7f1d7807dec0 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.186+0000 7f1d77fff700 1 --2- 172.21.15.17:0/1996514383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 0x7f1d7807c320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1d680098a0 tx=0x7f1d68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea2cc486999f099 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.186+0000 7f1d76ffd700 1 -- 172.21.15.17:0/1996514383 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d68004160 con 0x7f1d7807dec0 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.186+0000 7f1d76ffd700 1 -- 172.21.15.17:0/1996514383 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d68004300 con 0x7f1d7807dec0 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.186+0000 7f1d76ffd700 1 -- 172.21.15.17:0/1996514383 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d68011b50 con 0x7f1d7807dec0 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.187+0000 7f1d7e962700 1 -- 172.21.15.17:0/1996514383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 msgr2=0x7f1d7807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.187+0000 7f1d7e962700 1 --2- 172.21.15.17:0/1996514383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 0x7f1d7807c320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1d680098a0 tx=0x7f1d68009bb0 comp rx=0 tx=0).stop 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.187+0000 7f1d7e962700 1 -- 172.21.15.17:0/1996514383 shutdown_connections 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.187+0000 7f1d7e962700 1 --2- 172.21.15.17:0/1996514383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 0x7f1d7807c320 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:53.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.187+0000 7f1d7e962700 1 -- 172.21.15.17:0/1996514383 >> 172.21.15.17:0/1996514383 conn(0x7f1d781030b0 msgr2=0x7f1d781054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.187+0000 7f1d7e962700 1 -- 172.21.15.17:0/1996514383 shutdown_connections 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.187+0000 7f1d7e962700 1 -- 172.21.15.17:0/1996514383 wait complete. 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.188+0000 7f1d7e962700 1 Processor -- start 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.188+0000 7f1d7e962700 1 -- start start 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.188+0000 7f1d7e962700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 0x7f1d7819e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.188+0000 7f1d7e962700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7810e110 con 0x7f1d7807dec0 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.188+0000 7f1d77fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 0x7f1d7819e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.188+0000 7f1d77fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 0x7f1d7819e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38374/0 (socket says 172.21.15.17:38374) 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.188+0000 7f1d77fff700 1 -- 172.21.15.17:0/643500219 learned_addr learned my addr 172.21.15.17:0/643500219 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.188+0000 7f1d77fff700 1 -- 172.21.15.17:0/643500219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7819e570 con 0x7f1d7807dec0 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.189+0000 7f1d77fff700 1 --2- 172.21.15.17:0/643500219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 0x7f1d7819e030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1d68011850 tx=0x7f1d68011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.189+0000 7f1d757fa700 1 -- 172.21.15.17:0/643500219 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d68003af0 con 0x7f1d7807dec0 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.189+0000 7f1d757fa700 1 -- 172.21.15.17:0/643500219 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d68020070 con 0x7f1d7807dec0 2024-03-24T02:50:53.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.189+0000 7f1d7e962700 1 -- 172.21.15.17:0/643500219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7819e800 con 0x7f1d7807dec0 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.189+0000 7f1d757fa700 1 -- 172.21.15.17:0/643500219 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d680195a0 con 0x7f1d7807dec0 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.189+0000 7f1d7e962700 1 -- 172.21.15.17:0/643500219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d781a14f0 con 0x7f1d7807dec0 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.190+0000 7f1d757fa700 1 -- 172.21.15.17:0/643500219 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 3) v1 ==== 45989+0+0 (secure 0 0 0) 0x7f1d68019850 con 0x7f1d7807dec0 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.190+0000 7f1d7e962700 1 -- 172.21.15.17:0/643500219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d780637c0 con 0x7f1d7807dec0 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.190+0000 7f1d757fa700 1 --2- 172.21.15.17:0/643500219 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7f1d64038e00 0x7f1d6403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.190+0000 7f1d757fa700 1 -- 172.21.15.17:0/643500219 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1d68011e00 con 0x7f1d7807dec0 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.190+0000 7f1d777fe700 1 --2- 172.21.15.17:0/643500219 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7f1d64038e00 0x7f1d6403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.190+0000 7f1d777fe700 1 --2- 172.21.15.17:0/643500219 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7f1d64038e00 0x7f1d6403b2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1d60006fd0 tx=0x7f1d60006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.195+0000 7f1d757fa700 1 -- 172.21.15.17:0/643500219 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1d68003c90 con 0x7f1d7807dec0 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.299+0000 7f1d757fa700 1 -- 172.21.15.17:0/643500219 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f1d6800b430 con 0x7f1d7807dec0 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.488+0000 7f1d7e962700 1 -- 172.21.15.17:0/643500219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1d7802da30 con 0x7f1d7807dec0 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.489+0000 7f1d757fa700 1 -- 172.21.15.17:0/643500219 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f1d680439b0 con 0x7f1d7807dec0 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.491+0000 7f1d7e962700 1 -- 172.21.15.17:0/643500219 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7f1d64038e00 msgr2=0x7f1d6403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.491+0000 7f1d7e962700 1 --2- 172.21.15.17:0/643500219 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7f1d64038e00 0x7f1d6403b2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1d60006fd0 tx=0x7f1d60006e40 comp rx=0 tx=0).stop 2024-03-24T02:50:53.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.491+0000 7f1d7e962700 1 -- 172.21.15.17:0/643500219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 msgr2=0x7f1d7819e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:53.098 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.491+0000 7f1d7e962700 1 --2- 172.21.15.17:0/643500219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 0x7f1d7819e030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1d68011850 tx=0x7f1d68011c10 comp rx=0 tx=0).stop 2024-03-24T02:50:53.098 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.491+0000 7f1d7e962700 1 -- 172.21.15.17:0/643500219 shutdown_connections 2024-03-24T02:50:53.098 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.491+0000 7f1d7e962700 1 --2- 172.21.15.17:0/643500219 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7f1d64038e00 0x7f1d6403b2c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:53.098 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.491+0000 7f1d7e962700 1 --2- 172.21.15.17:0/643500219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d7807dec0 0x7f1d7819e030 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:53.098 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.491+0000 7f1d7e962700 1 -- 172.21.15.17:0/643500219 >> 172.21.15.17:0/643500219 conn(0x7f1d781030b0 msgr2=0x7f1d78103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:53.098 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.491+0000 7f1d7e962700 1 -- 172.21.15.17:0/643500219 shutdown_connections 2024-03-24T02:50:53.098 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:52.491+0000 7f1d7e962700 1 -- 172.21.15.17:0/643500219 wait complete. 2024-03-24T02:50:53.098 INFO:teuthology.orchestra.run.smithi017.stdout:mgr is available 2024-03-24T02:50:53.453 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:53 smithi017 ceph-mon[31926]: mgrmap e4: x(active, since 3s) 2024-03-24T02:50:53.454 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:53 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/643500219' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T02:50:54.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:54.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [global] 2024-03-24T02:50:54.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout fsid = 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:50:54.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-24T02:50:54.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-24T02:50:54.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-24T02:50:54.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-24T02:50:54.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:54.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-24T02:50:54.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-24T02:50:54.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [osd] 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.471+0000 7f5ae7828700 1 Processor -- start 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.471+0000 7f5ae7828700 1 -- start start 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.472+0000 7f5ae7828700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 0x7f5ae0107f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.472+0000 7f5ae7828700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae0108500 con 0x7f5ae0107b10 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.472+0000 7f5ae55c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 0x7f5ae0107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.472+0000 7f5ae55c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 0x7f5ae0107f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38390/0 (socket says 172.21.15.17:38390) 2024-03-24T02:50:54.309 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.472+0000 7f5ae55c4700 1 -- 172.21.15.17:0/470916826 learned_addr learned my addr 172.21.15.17:0/470916826 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.472+0000 7f5ae55c4700 1 -- 172.21.15.17:0/470916826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae0108d10 con 0x7f5ae0107b10 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.472+0000 7f5ae55c4700 1 --2- 172.21.15.17:0/470916826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 0x7f5ae0107f30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0098a0 tx=0x7f5adc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc41bb6f8dee119e server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.472+0000 7f5ad3fff700 1 -- 172.21.15.17:0/470916826 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5adc004160 con 0x7f5ae0107b10 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.472+0000 7f5ad3fff700 1 -- 172.21.15.17:0/470916826 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5adc004300 con 0x7f5ae0107b10 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.472+0000 7f5ad3fff700 1 -- 172.21.15.17:0/470916826 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5adc011b50 con 0x7f5ae0107b10 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.473+0000 7f5ae7828700 1 -- 172.21.15.17:0/470916826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 msgr2=0x7f5ae0107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.473+0000 7f5ae7828700 1 --2- 172.21.15.17:0/470916826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 0x7f5ae0107f30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0098a0 tx=0x7f5adc009bb0 comp rx=0 tx=0).stop 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.473+0000 7f5ae7828700 1 -- 172.21.15.17:0/470916826 shutdown_connections 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.473+0000 7f5ae7828700 1 --2- 172.21.15.17:0/470916826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 0x7f5ae0107f30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.473+0000 7f5ae7828700 1 -- 172.21.15.17:0/470916826 >> 172.21.15.17:0/470916826 conn(0x7f5ae01030d0 msgr2=0x7f5ae01054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.473+0000 7f5ae7828700 1 -- 172.21.15.17:0/470916826 shutdown_connections 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.473+0000 7f5ae7828700 1 -- 172.21.15.17:0/470916826 wait complete. 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.474+0000 7f5ae7828700 1 Processor -- start 2024-03-24T02:50:54.310 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.474+0000 7f5ae7828700 1 -- start start 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.474+0000 7f5ae7828700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 0x7f5ae019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.474+0000 7f5ae7828700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae010c0c0 con 0x7f5ae0107b10 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.474+0000 7f5ae55c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 0x7f5ae019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.474+0000 7f5ae55c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 0x7f5ae019be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38406/0 (socket says 172.21.15.17:38406) 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.474+0000 7f5ae55c4700 1 -- 172.21.15.17:0/603844654 learned_addr learned my addr 172.21.15.17:0/603844654 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.474+0000 7f5ae55c4700 1 -- 172.21.15.17:0/603844654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae019c3b0 con 0x7f5ae0107b10 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.474+0000 7f5ae55c4700 1 --2- 172.21.15.17:0/603844654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 0x7f5ae019be70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5adc011850 tx=0x7f5adc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.474+0000 7f5ad27fc700 1 -- 172.21.15.17:0/603844654 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5adc003af0 con 0x7f5ae0107b10 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.475+0000 7f5ad27fc700 1 -- 172.21.15.17:0/603844654 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5adc020070 con 0x7f5ae0107b10 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.475+0000 7f5ae7828700 1 -- 172.21.15.17:0/603844654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae019c640 con 0x7f5ae0107b10 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.475+0000 7f5ad27fc700 1 -- 172.21.15.17:0/603844654 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5adc0195a0 con 0x7f5ae0107b10 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.475+0000 7f5ae7828700 1 -- 172.21.15.17:0/603844654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae019f330 con 0x7f5ae0107b10 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.475+0000 7f5ad27fc700 1 -- 172.21.15.17:0/603844654 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f5adc0198a0 con 0x7f5ae0107b10 2024-03-24T02:50:54.311 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.476+0000 7f5ae7828700 1 -- 172.21.15.17:0/603844654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae00637c0 con 0x7f5ae0107b10 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.476+0000 7f5ad27fc700 1 --2- 172.21.15.17:0/603844654 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7f5acc038ed0 0x7f5acc03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.476+0000 7f5ae4dc3700 1 --2- 172.21.15.17:0/603844654 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7f5acc038ed0 0x7f5acc03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.476+0000 7f5ad27fc700 1 -- 172.21.15.17:0/603844654 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5adc011e00 con 0x7f5ae0107b10 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.476+0000 7f5ae4dc3700 1 --2- 172.21.15.17:0/603844654 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7f5acc038ed0 0x7f5acc03b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4006fd0 tx=0x7f5ad4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.481+0000 7f5ad27fc700 1 -- 172.21.15.17:0/603844654 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5adc003c90 con 0x7f5ae0107b10 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.685+0000 7f5ae7828700 1 -- 172.21.15.17:0/603844654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5ae01a0680 con 0x7f5ae0107b10 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.686+0000 7f5ad27fc700 1 -- 172.21.15.17:0/603844654 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f5adc04cc50 con 0x7f5ae0107b10 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.688+0000 7f5ae7828700 1 -- 172.21.15.17:0/603844654 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7f5acc038ed0 msgr2=0x7f5acc03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.688+0000 7f5ae7828700 1 --2- 172.21.15.17:0/603844654 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7f5acc038ed0 0x7f5acc03b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4006fd0 tx=0x7f5ad4006e40 comp rx=0 tx=0).stop 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.688+0000 7f5ae7828700 1 -- 172.21.15.17:0/603844654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 msgr2=0x7f5ae019be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.688+0000 7f5ae7828700 1 --2- 172.21.15.17:0/603844654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 0x7f5ae019be70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5adc011850 tx=0x7f5adc011c10 comp rx=0 tx=0).stop 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.688+0000 7f5ae7828700 1 -- 172.21.15.17:0/603844654 shutdown_connections 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.688+0000 7f5ae7828700 1 --2- 172.21.15.17:0/603844654 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7f5acc038ed0 0x7f5acc03b390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:54.312 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.688+0000 7f5ae7828700 1 --2- 172.21.15.17:0/603844654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ae0107b10 0x7f5ae019be70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:54.313 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.688+0000 7f5ae7828700 1 -- 172.21.15.17:0/603844654 >> 172.21.15.17:0/603844654 conn(0x7f5ae01030d0 msgr2=0x7f5ae0103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:54.313 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.688+0000 7f5ae7828700 1 -- 172.21.15.17:0/603844654 shutdown_connections 2024-03-24T02:50:54.313 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:53.688+0000 7f5ae7828700 1 -- 172.21.15.17:0/603844654 wait complete. 2024-03-24T02:50:54.313 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling cephadm module... 2024-03-24T02:50:54.644 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:54 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/603844654' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-24T02:50:55.604 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:55 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/778880746' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-24T02:50:55.605 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:55 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: ignoring --setuser ceph since I am not root 2024-03-24T02:50:55.605 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:55 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: ignoring --setgroup ceph since I am not root 2024-03-24T02:50:55.895 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.750+0000 7fa099845700 1 Processor -- start 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.750+0000 7fa099845700 1 -- start start 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.750+0000 7fa099845700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 0x7fa09410a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.750+0000 7fa099845700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09407be90 con 0x7fa094108580 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.751+0000 7fa092ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 0x7fa09410a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.751+0000 7fa092ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 0x7fa09410a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38422/0 (socket says 172.21.15.17:38422) 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.751+0000 7fa092ffd700 1 -- 172.21.15.17:0/4073387958 learned_addr learned my addr 172.21.15.17:0/4073387958 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.751+0000 7fa092ffd700 1 -- 172.21.15.17:0/4073387958 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09410aeb0 con 0x7fa094108580 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.751+0000 7fa092ffd700 1 --2- 172.21.15.17:0/4073387958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 0x7fa09410a970 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009e80 tx=0x7fa08c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a544129f9e7e05 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.751+0000 7fa091ffb700 1 -- 172.21.15.17:0/4073387958 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa08c017070 con 0x7fa094108580 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.751+0000 7fa091ffb700 1 -- 172.21.15.17:0/4073387958 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa08c00b8c0 con 0x7fa094108580 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.751+0000 7fa091ffb700 1 -- 172.21.15.17:0/4073387958 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa08c0045c0 con 0x7fa094108580 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.752+0000 7fa099845700 1 -- 172.21.15.17:0/4073387958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 msgr2=0x7fa09410a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.752+0000 7fa099845700 1 --2- 172.21.15.17:0/4073387958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 0x7fa09410a970 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009e80 tx=0x7fa08c00b330 comp rx=0 tx=0).stop 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.752+0000 7fa099845700 1 -- 172.21.15.17:0/4073387958 shutdown_connections 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.752+0000 7fa099845700 1 --2- 172.21.15.17:0/4073387958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 0x7fa09410a970 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.752+0000 7fa099845700 1 -- 172.21.15.17:0/4073387958 >> 172.21.15.17:0/4073387958 conn(0x7fa094101fb0 msgr2=0x7fa0941043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.752+0000 7fa099845700 1 -- 172.21.15.17:0/4073387958 shutdown_connections 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.752+0000 7fa099845700 1 -- 172.21.15.17:0/4073387958 wait complete. 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.752+0000 7fa099845700 1 Processor -- start 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.753+0000 7fa099845700 1 -- start start 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.753+0000 7fa099845700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 0x7fa094107e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.753+0000 7fa099845700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09410bee0 con 0x7fa094108580 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.753+0000 7fa092ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 0x7fa094107e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.753+0000 7fa092ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 0x7fa094107e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38424/0 (socket says 172.21.15.17:38424) 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.753+0000 7fa092ffd700 1 -- 172.21.15.17:0/778880746 learned_addr learned my addr 172.21.15.17:0/778880746 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.753+0000 7fa092ffd700 1 -- 172.21.15.17:0/778880746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa094108340 con 0x7fa094108580 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.753+0000 7fa092ffd700 1 --2- 172.21.15.17:0/778880746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 0x7fa094107e00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009530 tx=0x7fa08c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.753+0000 7fa098843700 1 -- 172.21.15.17:0/778880746 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa08c020020 con 0x7fa094108580 2024-03-24T02:50:55.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.754+0000 7fa098843700 1 -- 172.21.15.17:0/778880746 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa08c003b50 con 0x7fa094108580 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.754+0000 7fa099845700 1 -- 172.21.15.17:0/778880746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa094106530 con 0x7fa094108580 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.754+0000 7fa098843700 1 -- 172.21.15.17:0/778880746 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa08c017040 con 0x7fa094108580 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.754+0000 7fa099845700 1 -- 172.21.15.17:0/778880746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa094106a70 con 0x7fa094108580 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.754+0000 7fa098843700 1 -- 172.21.15.17:0/778880746 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7fa08c023020 con 0x7fa094108580 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.754+0000 7fa099845700 1 -- 172.21.15.17:0/778880746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0940637c0 con 0x7fa094108580 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.755+0000 7fa098843700 1 --2- 172.21.15.17:0/778880746 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fa080038ed0 0x7fa08003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.755+0000 7fa098843700 1 -- 172.21.15.17:0/778880746 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa08c00f050 con 0x7fa094108580 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.755+0000 7fa0927fc700 1 --2- 172.21.15.17:0/778880746 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fa080038ed0 0x7fa08003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.755+0000 7fa0927fc700 1 --2- 172.21.15.17:0/778880746 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fa080038ed0 0x7fa08003b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa07c006fd0 tx=0x7fa07c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:54.759+0000 7fa098843700 1 -- 172.21.15.17:0/778880746 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa08c02a020 con 0x7fa094108580 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.023+0000 7fa099845700 1 -- 172.21.15.17:0/778880746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fa094195540 con 0x7fa094108580 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.301+0000 7fa098843700 1 -- 172.21.15.17:0/778880746 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fa08c01b530 con 0x7fa094108580 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.303+0000 7fa098843700 1 -- 172.21.15.17:0/778880746 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fa08c025400 con 0x7fa094108580 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.305+0000 7fa099845700 1 -- 172.21.15.17:0/778880746 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fa080038ed0 msgr2=0x7fa08003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:55.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.305+0000 7fa099845700 1 --2- 172.21.15.17:0/778880746 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fa080038ed0 0x7fa08003b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa07c006fd0 tx=0x7fa07c006e40 comp rx=0 tx=0).stop 2024-03-24T02:50:55.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.305+0000 7fa099845700 1 -- 172.21.15.17:0/778880746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 msgr2=0x7fa094107e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:55.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.305+0000 7fa099845700 1 --2- 172.21.15.17:0/778880746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 0x7fa094107e00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009530 tx=0x7fa08c00bd50 comp rx=0 tx=0).stop 2024-03-24T02:50:55.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.305+0000 7fa099845700 1 -- 172.21.15.17:0/778880746 shutdown_connections 2024-03-24T02:50:55.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.305+0000 7fa099845700 1 --2- 172.21.15.17:0/778880746 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fa080038ed0 0x7fa08003b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:55.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.305+0000 7fa099845700 1 --2- 172.21.15.17:0/778880746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa094108580 0x7fa094107e00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:55.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.305+0000 7fa099845700 1 -- 172.21.15.17:0/778880746 >> 172.21.15.17:0/778880746 conn(0x7fa094101fb0 msgr2=0x7fa094102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:55.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.305+0000 7fa099845700 1 -- 172.21.15.17:0/778880746 shutdown_connections 2024-03-24T02:50:55.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:55.305+0000 7fa099845700 1 -- 172.21.15.17:0/778880746 wait complete. 2024-03-24T02:50:56.183 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:55 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:55.871+0000 7ff362a75200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T02:50:56.183 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:55 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:55.970+0000 7ff362a75200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T02:50:56.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:56 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/778880746' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-24T02:50:56.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:56 smithi017 ceph-mon[31926]: mgrmap e5: x(active, since 6s) 2024-03-24T02:50:56.878 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:56.638+0000 7ff362a75200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T02:50:56.879 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:56.733+0000 7ff362a75200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T02:50:56.879 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:56.878+0000 7ff362a75200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T02:50:57.178 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:56.976+0000 7ff362a75200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T02:50:57.178 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:57.060+0000 7ff362a75200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T02:50:57.186 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.282+0000 7fe45744e700 1 Processor -- start 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.282+0000 7fe45744e700 1 -- start start 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.283+0000 7fe45744e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 0x7fe450109510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.283+0000 7fe45744e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe450109ae0 con 0x7fe4501090f0 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.283+0000 7fe4551ea700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 0x7fe450109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.283+0000 7fe4551ea700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 0x7fe450109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38458/0 (socket says 172.21.15.17:38458) 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.283+0000 7fe4551ea700 1 -- 172.21.15.17:0/1448008564 learned_addr learned my addr 172.21.15.17:0/1448008564 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:57.187 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.283+0000 7fe4551ea700 1 -- 172.21.15.17:0/1448008564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe450109c60 con 0x7fe4501090f0 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.283+0000 7fe4551ea700 1 --2- 172.21.15.17:0/1448008564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 0x7fe450109510 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe4440098a0 tx=0x7fe444009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e75d23d58a6083d server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.283+0000 7fe443fff700 1 -- 172.21.15.17:0/1448008564 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe444004160 con 0x7fe4501090f0 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.283+0000 7fe443fff700 1 -- 172.21.15.17:0/1448008564 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe444004300 con 0x7fe4501090f0 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.283+0000 7fe443fff700 1 -- 172.21.15.17:0/1448008564 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe444011b50 con 0x7fe4501090f0 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.284+0000 7fe45744e700 1 -- 172.21.15.17:0/1448008564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 msgr2=0x7fe450109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.284+0000 7fe45744e700 1 --2- 172.21.15.17:0/1448008564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 0x7fe450109510 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe4440098a0 tx=0x7fe444009bb0 comp rx=0 tx=0).stop 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.284+0000 7fe45744e700 1 -- 172.21.15.17:0/1448008564 shutdown_connections 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.284+0000 7fe45744e700 1 --2- 172.21.15.17:0/1448008564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 0x7fe450109510 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.284+0000 7fe45744e700 1 -- 172.21.15.17:0/1448008564 >> 172.21.15.17:0/1448008564 conn(0x7fe4501017d0 msgr2=0x7fe450103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.284+0000 7fe45744e700 1 -- 172.21.15.17:0/1448008564 shutdown_connections 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.284+0000 7fe45744e700 1 -- 172.21.15.17:0/1448008564 wait complete. 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.285+0000 7fe45744e700 1 Processor -- start 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.285+0000 7fe45744e700 1 -- start start 2024-03-24T02:50:57.188 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.285+0000 7fe45744e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 0x7fe45019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.285+0000 7fe45744e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe450106950 con 0x7fe4501090f0 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.285+0000 7fe4551ea700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 0x7fe45019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.285+0000 7fe4551ea700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 0x7fe45019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38474/0 (socket says 172.21.15.17:38474) 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.285+0000 7fe4551ea700 1 -- 172.21.15.17:0/3006334661 learned_addr learned my addr 172.21.15.17:0/3006334661 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.285+0000 7fe4551ea700 1 -- 172.21.15.17:0/3006334661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45019c000 con 0x7fe4501090f0 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.286+0000 7fe4551ea700 1 --2- 172.21.15.17:0/3006334661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 0x7fe45019bac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe444011850 tx=0x7fe444011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.286+0000 7fe4427fc700 1 -- 172.21.15.17:0/3006334661 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe444003af0 con 0x7fe4501090f0 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.286+0000 7fe45744e700 1 -- 172.21.15.17:0/3006334661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45019c2f0 con 0x7fe4501090f0 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.286+0000 7fe4427fc700 1 -- 172.21.15.17:0/3006334661 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe444020070 con 0x7fe4501090f0 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.286+0000 7fe4427fc700 1 -- 172.21.15.17:0/3006334661 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe444019550 con 0x7fe4501090f0 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.286+0000 7fe45744e700 1 -- 172.21.15.17:0/3006334661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45019ef80 con 0x7fe4501090f0 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.287+0000 7fe45744e700 1 -- 172.21.15.17:0/3006334661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4500637c0 con 0x7fe4501090f0 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.287+0000 7fe4427fc700 1 -- 172.21.15.17:0/3006334661 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fe444019850 con 0x7fe4501090f0 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.288+0000 7fe4427fc700 1 --2- 172.21.15.17:0/3006334661 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fe43c038f20 0x7fe43c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:50:57.189 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.288+0000 7fe4427fc700 1 -- 172.21.15.17:0/3006334661 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe444011e00 con 0x7fe4501090f0 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.288+0000 7fe4549e9700 1 -- 172.21.15.17:0/3006334661 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fe43c038f20 msgr2=0x7fe43c03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/4291784529 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.288+0000 7fe4549e9700 1 --2- 172.21.15.17:0/3006334661 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fe43c038f20 0x7fe43c03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.296+0000 7fe4427fc700 1 -- 172.21.15.17:0/3006334661 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe444019b40 con 0x7fe4501090f0 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.489+0000 7fe4549e9700 1 -- 172.21.15.17:0/3006334661 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fe43c038f20 msgr2=0x7fe43c03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/4291784529 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.489+0000 7fe4549e9700 1 --2- 172.21.15.17:0/3006334661 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fe43c038f20 0x7fe43c03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.562+0000 7fe45744e700 1 -- 172.21.15.17:0/3006334661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe45019f310 con 0x7fe4501090f0 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.562+0000 7fe4427fc700 1 -- 172.21.15.17:0/3006334661 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe444022d90 con 0x7fe4501090f0 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.565+0000 7fe45744e700 1 -- 172.21.15.17:0/3006334661 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fe43c038f20 msgr2=0x7fe43c03b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.565+0000 7fe45744e700 1 --2- 172.21.15.17:0/3006334661 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fe43c038f20 0x7fe43c03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.565+0000 7fe45744e700 1 -- 172.21.15.17:0/3006334661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 msgr2=0x7fe45019bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.565+0000 7fe45744e700 1 --2- 172.21.15.17:0/3006334661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 0x7fe45019bac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe444011850 tx=0x7fe444011c10 comp rx=0 tx=0).stop 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.565+0000 7fe45744e700 1 -- 172.21.15.17:0/3006334661 shutdown_connections 2024-03-24T02:50:57.190 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.565+0000 7fe45744e700 1 --2- 172.21.15.17:0/3006334661 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fe43c038f20 0x7fe43c03b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:57.191 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.565+0000 7fe45744e700 1 --2- 172.21.15.17:0/3006334661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4501090f0 0x7fe45019bac0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:50:57.191 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.565+0000 7fe45744e700 1 -- 172.21.15.17:0/3006334661 >> 172.21.15.17:0/3006334661 conn(0x7fe4501017d0 msgr2=0x7fe450102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:50:57.191 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.565+0000 7fe45744e700 1 -- 172.21.15.17:0/3006334661 shutdown_connections 2024-03-24T02:50:57.191 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:56.565+0000 7fe45744e700 1 -- 172.21.15.17:0/3006334661 wait complete. 2024-03-24T02:50:57.191 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for the mgr to restart... 2024-03-24T02:50:57.191 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mgr epoch 5... 2024-03-24T02:50:57.445 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:50:57 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3006334661' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-24T02:50:57.445 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:57.331+0000 7ff362a75200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T02:50:57.445 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:57.444+0000 7ff362a75200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T02:50:57.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:57.534+0000 7ff362a75200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T02:50:57.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:57.633+0000 7ff362a75200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T02:50:58.206 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:57.814+0000 7ff362a75200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T02:50:59.082 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:58 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:58.802+0000 7ff362a75200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T02:50:59.082 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:58 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:58.941+0000 7ff362a75200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T02:50:59.376 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:59 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:59.282+0000 7ff362a75200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T02:50:59.704 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:59 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:59.376+0000 7ff362a75200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T02:50:59.704 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:59 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:59.578+0000 7ff362a75200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T02:50:59.956 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:50:59 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:50:59.769+0000 7ff362a75200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T02:51:00.319 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:00.059+0000 7ff362a75200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T02:51:00.685 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:00.318+0000 7ff362a75200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T02:51:00.686 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:00.560+0000 7ff362a75200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T02:51:01.705 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:01 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:01.381+0000 7ff362a75200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T02:51:02.278 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:01 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:01.958+0000 7ff362a75200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T02:51:02.278 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:02.057+0000 7ff362a75200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T02:51:02.278 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:02.278+0000 7ff362a75200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T02:51:02.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: Active manager daemon x restarted 2024-03-24T02:51:02.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: Activating manager daemon x 2024-03-24T02:51:02.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: osdmap e2: 0 total, 0 up, 0 in 2024-03-24T02:51:02.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: mgrmap e6: x(active, starting, since 0.00352396s) 2024-03-24T02:51:02.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:51:02.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T02:51:02.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T02:51:02.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T02:51:02.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T02:51:02.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: Manager daemon x is now available 2024-03-24T02:51:02.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:02.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:02.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:51:02.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:02 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:51:03.601 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:03 smithi017 ceph-mon[31926]: Found migration_current of "None". Setting to last migration. 2024-03-24T02:51:03.601 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:03 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:51:03.601 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:03 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-24T02:51:03.601 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:03 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-24T02:51:03.601 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:03 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:03.601 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:03 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:03.601 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:03 smithi017 ceph-mon[31926]: mgrmap e7: x(active, since 1.00543s) 2024-03-24T02:51:03.937 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-03-24T02:51:03.937 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-24T02:51:03.937 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-24T02:51:03.937 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-03-24T02:51:03.937 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.674+0000 7fd579f41700 1 Processor -- start 2024-03-24T02:51:03.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.674+0000 7fd579f41700 1 -- start start 2024-03-24T02:51:03.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.674+0000 7fd579f41700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 0x7fd57410a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:03.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.674+0000 7fd579f41700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57407be30 con 0x7fd5741085f0 2024-03-24T02:51:03.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.674+0000 7fd5737fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 0x7fd57410a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:03.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.674+0000 7fd5737fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 0x7fd57410a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38490/0 (socket says 172.21.15.17:38490) 2024-03-24T02:51:03.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.674+0000 7fd5737fe700 1 -- 172.21.15.17:0/4092281894 learned_addr learned my addr 172.21.15.17:0/4092281894 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:03.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.674+0000 7fd5737fe700 1 -- 172.21.15.17:0/4092281894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57410af20 con 0x7fd5741085f0 2024-03-24T02:51:03.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.675+0000 7fd5737fe700 1 --2- 172.21.15.17:0/4092281894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 0x7fd57410a9e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd5640098a0 tx=0x7fd564009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe284ab01614028 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:03.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.675+0000 7fd5727fc700 1 -- 172.21.15.17:0/4092281894 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd564004160 con 0x7fd5741085f0 2024-03-24T02:51:03.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.675+0000 7fd5727fc700 1 -- 172.21.15.17:0/4092281894 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd564004300 con 0x7fd5741085f0 2024-03-24T02:51:03.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.675+0000 7fd5727fc700 1 -- 172.21.15.17:0/4092281894 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd564011b50 con 0x7fd5741085f0 2024-03-24T02:51:03.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.675+0000 7fd579f41700 1 -- 172.21.15.17:0/4092281894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 msgr2=0x7fd57410a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.675+0000 7fd579f41700 1 --2- 172.21.15.17:0/4092281894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 0x7fd57410a9e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd5640098a0 tx=0x7fd564009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.675+0000 7fd579f41700 1 -- 172.21.15.17:0/4092281894 shutdown_connections 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.675+0000 7fd579f41700 1 --2- 172.21.15.17:0/4092281894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 0x7fd57410a9e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.675+0000 7fd579f41700 1 -- 172.21.15.17:0/4092281894 >> 172.21.15.17:0/4092281894 conn(0x7fd574101fe0 msgr2=0x7fd574104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.676+0000 7fd579f41700 1 -- 172.21.15.17:0/4092281894 shutdown_connections 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.676+0000 7fd579f41700 1 -- 172.21.15.17:0/4092281894 wait complete. 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.676+0000 7fd579f41700 1 Processor -- start 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.676+0000 7fd579f41700 1 -- start start 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.677+0000 7fd579f41700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 0x7fd57419be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.677+0000 7fd579f41700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57410bef0 con 0x7fd5741085f0 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.677+0000 7fd5737fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 0x7fd57419be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.677+0000 7fd5737fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 0x7fd57419be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38498/0 (socket says 172.21.15.17:38498) 2024-03-24T02:51:03.939 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.677+0000 7fd5737fe700 1 -- 172.21.15.17:0/2977939103 learned_addr learned my addr 172.21.15.17:0/2977939103 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.677+0000 7fd5737fe700 1 -- 172.21.15.17:0/2977939103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57419c3a0 con 0x7fd5741085f0 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.677+0000 7fd5737fe700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 0x7fd57419be60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd564011b60 tx=0x7fd564011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.677+0000 7fd570ff9700 1 -- 172.21.15.17:0/2977939103 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd564003970 con 0x7fd5741085f0 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.677+0000 7fd570ff9700 1 -- 172.21.15.17:0/2977939103 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd564003b10 con 0x7fd5741085f0 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.677+0000 7fd579f41700 1 -- 172.21.15.17:0/2977939103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57419c630 con 0x7fd5741085f0 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.677+0000 7fd570ff9700 1 -- 172.21.15.17:0/2977939103 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd564024610 con 0x7fd5741085f0 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.677+0000 7fd579f41700 1 -- 172.21.15.17:0/2977939103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57419f2d0 con 0x7fd5741085f0 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.678+0000 7fd570ff9700 1 -- 172.21.15.17:0/2977939103 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fd564024910 con 0x7fd5741085f0 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.678+0000 7fd570ff9700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 0x7fd55c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.678+0000 7fd579f41700 1 -- 172.21.15.17:0/2977939103 --> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5740637c0 con 0x7fd55c038f20 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.678+0000 7fd570ff9700 1 -- 172.21.15.17:0/2977939103 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd564011e00 con 0x7fd5741085f0 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.678+0000 7fd572ffd700 1 -- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 msgr2=0x7fd55c03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/4291784529 2024-03-24T02:51:03.940 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.678+0000 7fd572ffd700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 0x7fd55c03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T02:51:03.941 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.879+0000 7fd572ffd700 1 -- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 msgr2=0x7fd55c03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/4291784529 2024-03-24T02:51:03.941 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:57.879+0000 7fd572ffd700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 0x7fd55c03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T02:51:03.941 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:58.280+0000 7fd572ffd700 1 -- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 msgr2=0x7fd55c03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/4291784529 2024-03-24T02:51:03.941 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:58.280+0000 7fd572ffd700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 0x7fd55c03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-24T02:51:03.941 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:59.081+0000 7fd572ffd700 1 -- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 msgr2=0x7fd55c03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/4291784529 2024-03-24T02:51:03.941 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:50:59.081+0000 7fd572ffd700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 0x7fd55c03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-24T02:51:03.941 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:00.682+0000 7fd572ffd700 1 -- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 msgr2=0x7fd55c03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/4291784529 2024-03-24T02:51:03.941 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:00.682+0000 7fd572ffd700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 0x7fd55c03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-24T02:51:03.941 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:02.293+0000 7fd570ff9700 1 -- 172.21.15.17:0/2977939103 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mgrmap(e 6) v1 ==== 45873+0+0 (secure 0 0 0) 0x7fd56402d910 con 0x7fd5741085f0 2024-03-24T02:51:03.941 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:02.293+0000 7fd570ff9700 1 -- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 msgr2=0x7fd55c03b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T02:51:03.941 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:02.293+0000 7fd570ff9700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 0x7fd55c03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:03.941 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.295+0000 7fd570ff9700 1 -- 172.21.15.17:0/2977939103 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 7) v1 ==== 46000+0+0 (secure 0 0 0) 0x7fd56404efe0 con 0x7fd5741085f0 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.295+0000 7fd570ff9700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fd55c03c620 0x7fd55c03ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.295+0000 7fd570ff9700 1 -- 172.21.15.17:0/2977939103 --> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5640214e0 con 0x7fd55c03c620 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.295+0000 7fd572ffd700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fd55c03c620 0x7fd55c03ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.296+0000 7fd572ffd700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fd55c03c620 0x7fd55c03ea10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd568003a10 tx=0x7fd568009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.297+0000 7fd570ff9700 1 -- 172.21.15.17:0/2977939103 <== mgr.14120 v2:172.21.15.17:6800/3007657427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fd5640214e0 con 0x7fd55c03c620 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.303+0000 7fd579f41700 1 -- 172.21.15.17:0/2977939103 --> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd574075a10 con 0x7fd55c03c620 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.304+0000 7fd570ff9700 1 -- 172.21.15.17:0/2977939103 <== mgr.14120 v2:172.21.15.17:6800/3007657427 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd574075a10 con 0x7fd55c03c620 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.304+0000 7fd579f41700 1 -- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fd55c03c620 msgr2=0x7fd55c03ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.304+0000 7fd579f41700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fd55c03c620 0x7fd55c03ea10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd568003a10 tx=0x7fd568009290 comp rx=0 tx=0).stop 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.304+0000 7fd579f41700 1 -- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 msgr2=0x7fd57419be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.304+0000 7fd579f41700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 0x7fd57419be60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd564011b60 tx=0x7fd564011c40 comp rx=0 tx=0).stop 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.304+0000 7fd579f41700 1 -- 172.21.15.17:0/2977939103 shutdown_connections 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.304+0000 7fd579f41700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fd55c03c620 0x7fd55c03ea10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:03.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.304+0000 7fd579f41700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5741085f0 0x7fd57419be60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:03.943 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.304+0000 7fd579f41700 1 --2- 172.21.15.17:0/2977939103 >> [v2:172.21.15.17:6800/4291784529,v1:172.21.15.17:6801/4291784529] conn(0x7fd55c038f20 0x7fd55c03b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:03.943 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.304+0000 7fd579f41700 1 -- 172.21.15.17:0/2977939103 >> 172.21.15.17:0/2977939103 conn(0x7fd574101fe0 msgr2=0x7fd574102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:03.943 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.304+0000 7fd579f41700 1 -- 172.21.15.17:0/2977939103 shutdown_connections 2024-03-24T02:51:03.943 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:03.304+0000 7fd579f41700 1 -- 172.21.15.17:0/2977939103 wait complete. 2024-03-24T02:51:03.943 INFO:teuthology.orchestra.run.smithi017.stdout:mgr epoch 5 is available 2024-03-24T02:51:03.943 INFO:teuthology.orchestra.run.smithi017.stdout:Setting orchestrator backend to cephadm... 2024-03-24T02:51:04.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:04 smithi017 ceph-mon[31926]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-24T02:51:04.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:04 smithi017 ceph-mon[31926]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-24T02:51:04.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:04 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:51:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.378+0000 7fa99d946700 1 Processor -- start 2024-03-24T02:51:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.378+0000 7fa99d946700 1 -- start start 2024-03-24T02:51:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.378+0000 7fa99d946700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 0x7fa998107f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.378+0000 7fa99d946700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981084e0 con 0x7fa998107af0 2024-03-24T02:51:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.378+0000 7fa996ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 0x7fa998107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.378+0000 7fa996ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 0x7fa998107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42608/0 (socket says 172.21.15.17:42608) 2024-03-24T02:51:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.378+0000 7fa996ffd700 1 -- 172.21.15.17:0/2149161980 learned_addr learned my addr 172.21.15.17:0/2149161980 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.378+0000 7fa996ffd700 1 -- 172.21.15.17:0/2149161980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa998108cf0 con 0x7fa998107af0 2024-03-24T02:51:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.379+0000 7fa996ffd700 1 --2- 172.21.15.17:0/2149161980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 0x7fa998107f10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa9880098a0 tx=0x7fa988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a231295a8a61f738 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.379+0000 7fa995ffb700 1 -- 172.21.15.17:0/2149161980 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa988004160 con 0x7fa998107af0 2024-03-24T02:51:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.379+0000 7fa995ffb700 1 -- 172.21.15.17:0/2149161980 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa988004300 con 0x7fa998107af0 2024-03-24T02:51:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.379+0000 7fa995ffb700 1 -- 172.21.15.17:0/2149161980 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa988011b50 con 0x7fa998107af0 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.379+0000 7fa99d946700 1 -- 172.21.15.17:0/2149161980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 msgr2=0x7fa998107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.379+0000 7fa99d946700 1 --2- 172.21.15.17:0/2149161980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 0x7fa998107f10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa9880098a0 tx=0x7fa988009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.379+0000 7fa99d946700 1 -- 172.21.15.17:0/2149161980 shutdown_connections 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.379+0000 7fa99d946700 1 --2- 172.21.15.17:0/2149161980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 0x7fa998107f10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.379+0000 7fa99d946700 1 -- 172.21.15.17:0/2149161980 >> 172.21.15.17:0/2149161980 conn(0x7fa9981030d0 msgr2=0x7fa9981054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.379+0000 7fa99d946700 1 -- 172.21.15.17:0/2149161980 shutdown_connections 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.379+0000 7fa99d946700 1 -- 172.21.15.17:0/2149161980 wait complete. 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.380+0000 7fa99d946700 1 Processor -- start 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.380+0000 7fa99d946700 1 -- start start 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.380+0000 7fa99d946700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 0x7fa99819be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.380+0000 7fa99d946700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99810c0a0 con 0x7fa998107af0 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.380+0000 7fa996ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 0x7fa99819be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.380+0000 7fa996ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 0x7fa99819be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42616/0 (socket says 172.21.15.17:42616) 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.380+0000 7fa996ffd700 1 -- 172.21.15.17:0/2175692401 learned_addr learned my addr 172.21.15.17:0/2175692401 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:05.217 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.381+0000 7fa996ffd700 1 -- 172.21.15.17:0/2175692401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99819c340 con 0x7fa998107af0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.381+0000 7fa996ffd700 1 --2- 172.21.15.17:0/2175692401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 0x7fa99819be00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa988009fb0 tx=0x7fa988011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.381+0000 7fa99c944700 1 -- 172.21.15.17:0/2175692401 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9880038b0 con 0x7fa998107af0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.381+0000 7fa99c944700 1 -- 172.21.15.17:0/2175692401 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa988011e50 con 0x7fa998107af0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.381+0000 7fa99d946700 1 -- 172.21.15.17:0/2175692401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99819c5d0 con 0x7fa998107af0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.381+0000 7fa99c944700 1 -- 172.21.15.17:0/2175692401 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa988019550 con 0x7fa998107af0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.381+0000 7fa99d946700 1 -- 172.21.15.17:0/2175692401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99819f2c0 con 0x7fa998107af0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.382+0000 7fa99c944700 1 -- 172.21.15.17:0/2175692401 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fa988019800 con 0x7fa998107af0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.382+0000 7fa99d946700 1 -- 172.21.15.17:0/2175692401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9980637c0 con 0x7fa998107af0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.382+0000 7fa99c944700 1 --2- 172.21.15.17:0/2175692401 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fa980038f20 0x7fa98003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.382+0000 7fa99c944700 1 -- 172.21.15.17:0/2175692401 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa98804d0d0 con 0x7fa998107af0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.382+0000 7fa9967fc700 1 --2- 172.21.15.17:0/2175692401 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fa980038f20 0x7fa98003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.383+0000 7fa9967fc700 1 --2- 172.21.15.17:0/2175692401 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fa980038f20 0x7fa98003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa98c006fd0 tx=0x7fa98c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.387+0000 7fa99c944700 1 -- 172.21.15.17:0/2175692401 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa98802d440 con 0x7fa998107af0 2024-03-24T02:51:05.218 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.608+0000 7fa99d946700 1 -- 172.21.15.17:0/2175692401 --> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fa99802daa0 con 0x7fa980038f20 2024-03-24T02:51:05.219 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.614+0000 7fa99c944700 1 -- 172.21.15.17:0/2175692401 <== mgr.14120 v2:172.21.15.17:6800/3007657427 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa99802daa0 con 0x7fa980038f20 2024-03-24T02:51:05.219 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.616+0000 7fa99d946700 1 -- 172.21.15.17:0/2175692401 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fa980038f20 msgr2=0x7fa98003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:05.219 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.616+0000 7fa99d946700 1 --2- 172.21.15.17:0/2175692401 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fa980038f20 0x7fa98003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa98c006fd0 tx=0x7fa98c006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:05.219 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.616+0000 7fa99d946700 1 -- 172.21.15.17:0/2175692401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 msgr2=0x7fa99819be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:05.219 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.616+0000 7fa99d946700 1 --2- 172.21.15.17:0/2175692401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 0x7fa99819be00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa988009fb0 tx=0x7fa988011bf0 comp rx=0 tx=0).stop 2024-03-24T02:51:05.219 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.616+0000 7fa99d946700 1 -- 172.21.15.17:0/2175692401 shutdown_connections 2024-03-24T02:51:05.219 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.616+0000 7fa99d946700 1 --2- 172.21.15.17:0/2175692401 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fa980038f20 0x7fa98003b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:05.219 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.616+0000 7fa99d946700 1 --2- 172.21.15.17:0/2175692401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa998107af0 0x7fa99819be00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:05.219 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.616+0000 7fa99d946700 1 -- 172.21.15.17:0/2175692401 >> 172.21.15.17:0/2175692401 conn(0x7fa9981030d0 msgr2=0x7fa998103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:05.219 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.616+0000 7fa99d946700 1 -- 172.21.15.17:0/2175692401 shutdown_connections 2024-03-24T02:51:05.219 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:04.616+0000 7fa99d946700 1 -- 172.21.15.17:0/2175692401 wait complete. 2024-03-24T02:51:05.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:05 smithi017 ceph-mon[31926]: [24/Mar/2024:02:51:03] ENGINE Bus STARTING 2024-03-24T02:51:05.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:05 smithi017 ceph-mon[31926]: [24/Mar/2024:02:51:04] ENGINE Serving on https://172.21.15.17:7150 2024-03-24T02:51:05.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:05 smithi017 ceph-mon[31926]: [24/Mar/2024:02:51:04] ENGINE Serving on http://172.21.15.17:8765 2024-03-24T02:51:05.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:05 smithi017 ceph-mon[31926]: [24/Mar/2024:02:51:04] ENGINE Bus STARTED 2024-03-24T02:51:05.536 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:05 smithi017 ceph-mon[31926]: mgrmap e8: x(active, since 2s) 2024-03-24T02:51:05.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:05 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:05.537 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:05 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:51:06.465 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:06 smithi017 ceph-mon[31926]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:51:06.473 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-24T02:51:06.473 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.646+0000 7fa11c4e4700 1 Processor -- start 2024-03-24T02:51:06.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.646+0000 7fa11c4e4700 1 -- start start 2024-03-24T02:51:06.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.646+0000 7fa11c4e4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 0x7fa11410aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:06.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.646+0000 7fa11c4e4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11407bdb0 con 0x7fa114108660 2024-03-24T02:51:06.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.646+0000 7fa11a280700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 0x7fa11410aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:06.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.646+0000 7fa11a280700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 0x7fa11410aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42622/0 (socket says 172.21.15.17:42622) 2024-03-24T02:51:06.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.646+0000 7fa11a280700 1 -- 172.21.15.17:0/1792626314 learned_addr learned my addr 172.21.15.17:0/1792626314 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:06.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.646+0000 7fa11a280700 1 -- 172.21.15.17:0/1792626314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11410af90 con 0x7fa114108660 2024-03-24T02:51:06.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.647+0000 7fa11a280700 1 --2- 172.21.15.17:0/1792626314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 0x7fa11410aa50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa1040098a0 tx=0x7fa104009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7dda0130efdcee0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:06.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.647+0000 7fa11927e700 1 -- 172.21.15.17:0/1792626314 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa104004160 con 0x7fa114108660 2024-03-24T02:51:06.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.647+0000 7fa11927e700 1 -- 172.21.15.17:0/1792626314 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa104004300 con 0x7fa114108660 2024-03-24T02:51:06.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.647+0000 7fa11927e700 1 -- 172.21.15.17:0/1792626314 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa104011b50 con 0x7fa114108660 2024-03-24T02:51:06.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.647+0000 7fa11c4e4700 1 -- 172.21.15.17:0/1792626314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 msgr2=0x7fa11410aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.647+0000 7fa11c4e4700 1 --2- 172.21.15.17:0/1792626314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 0x7fa11410aa50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa1040098a0 tx=0x7fa104009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.647+0000 7fa11c4e4700 1 -- 172.21.15.17:0/1792626314 shutdown_connections 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.647+0000 7fa11c4e4700 1 --2- 172.21.15.17:0/1792626314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 0x7fa11410aa50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.647+0000 7fa11c4e4700 1 -- 172.21.15.17:0/1792626314 >> 172.21.15.17:0/1792626314 conn(0x7fa114101fb0 msgr2=0x7fa1141043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.647+0000 7fa11c4e4700 1 -- 172.21.15.17:0/1792626314 shutdown_connections 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.647+0000 7fa11c4e4700 1 -- 172.21.15.17:0/1792626314 wait complete. 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.648+0000 7fa11c4e4700 1 Processor -- start 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.648+0000 7fa11c4e4700 1 -- start start 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.648+0000 7fa11c4e4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 0x7fa11419be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.648+0000 7fa11c4e4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11410bef0 con 0x7fa114108660 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.648+0000 7fa11a280700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 0x7fa11419be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.648+0000 7fa11a280700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 0x7fa11419be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42636/0 (socket says 172.21.15.17:42636) 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.648+0000 7fa11a280700 1 -- 172.21.15.17:0/4201777969 learned_addr learned my addr 172.21.15.17:0/4201777969 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.649+0000 7fa11a280700 1 -- 172.21.15.17:0/4201777969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11419c390 con 0x7fa114108660 2024-03-24T02:51:06.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.649+0000 7fa11a280700 1 --2- 172.21.15.17:0/4201777969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 0x7fa11419be50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa104011b60 tx=0x7fa104011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.649+0000 7fa10b7fe700 1 -- 172.21.15.17:0/4201777969 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa104003970 con 0x7fa114108660 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.649+0000 7fa11c4e4700 1 -- 172.21.15.17:0/4201777969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11419c620 con 0x7fa114108660 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.649+0000 7fa10b7fe700 1 -- 172.21.15.17:0/4201777969 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa104003b10 con 0x7fa114108660 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.649+0000 7fa10b7fe700 1 -- 172.21.15.17:0/4201777969 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa104024610 con 0x7fa114108660 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.649+0000 7fa11c4e4700 1 -- 172.21.15.17:0/4201777969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11419f310 con 0x7fa114108660 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.650+0000 7fa10b7fe700 1 -- 172.21.15.17:0/4201777969 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fa104024910 con 0x7fa114108660 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.650+0000 7fa11c4e4700 1 -- 172.21.15.17:0/4201777969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1140637c0 con 0x7fa114108660 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.650+0000 7fa10b7fe700 1 --2- 172.21.15.17:0/4201777969 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fa100038f20 0x7fa10003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.650+0000 7fa10b7fe700 1 -- 172.21.15.17:0/4201777969 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa104011e00 con 0x7fa114108660 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.650+0000 7fa119a7f700 1 --2- 172.21.15.17:0/4201777969 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fa100038f20 0x7fa10003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.651+0000 7fa119a7f700 1 --2- 172.21.15.17:0/4201777969 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fa100038f20 0x7fa10003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa110006fd0 tx=0x7fa110006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.655+0000 7fa10b7fe700 1 -- 172.21.15.17:0/4201777969 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa10400ba00 con 0x7fa114108660 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.862+0000 7fa11c4e4700 1 -- 172.21.15.17:0/4201777969 --> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fa11402daa0 con 0x7fa100038f20 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.862+0000 7fa10b7fe700 1 -- 172.21.15.17:0/4201777969 <== mgr.14120 v2:172.21.15.17:6800/3007657427 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fa11402daa0 con 0x7fa100038f20 2024-03-24T02:51:06.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.864+0000 7fa11c4e4700 1 -- 172.21.15.17:0/4201777969 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fa100038f20 msgr2=0x7fa10003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:06.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.864+0000 7fa11c4e4700 1 --2- 172.21.15.17:0/4201777969 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fa100038f20 0x7fa10003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa110006fd0 tx=0x7fa110006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:06.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.864+0000 7fa11c4e4700 1 -- 172.21.15.17:0/4201777969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 msgr2=0x7fa11419be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:06.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.864+0000 7fa11c4e4700 1 --2- 172.21.15.17:0/4201777969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 0x7fa11419be50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa104011b60 tx=0x7fa104011c40 comp rx=0 tx=0).stop 2024-03-24T02:51:06.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.864+0000 7fa11c4e4700 1 -- 172.21.15.17:0/4201777969 shutdown_connections 2024-03-24T02:51:06.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.864+0000 7fa11c4e4700 1 --2- 172.21.15.17:0/4201777969 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fa100038f20 0x7fa10003b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:06.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.864+0000 7fa11c4e4700 1 --2- 172.21.15.17:0/4201777969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa114108660 0x7fa11419be50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:06.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.864+0000 7fa11c4e4700 1 -- 172.21.15.17:0/4201777969 >> 172.21.15.17:0/4201777969 conn(0x7fa114101fb0 msgr2=0x7fa114102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:06.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.865+0000 7fa11c4e4700 1 -- 172.21.15.17:0/4201777969 shutdown_connections 2024-03-24T02:51:06.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:05.865+0000 7fa11c4e4700 1 -- 172.21.15.17:0/4201777969 wait complete. 2024-03-24T02:51:06.477 INFO:teuthology.orchestra.run.smithi017.stdout:Generating ssh key... 2024-03-24T02:51:07.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-mon[31926]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:51:07.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:07.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:07.456 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: Generating public/private rsa key pair. 2024-03-24T02:51:07.456 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: Your identification has been saved in /tmp/tmp3ii2jujb/key. 2024-03-24T02:51:07.456 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: Your public key has been saved in /tmp/tmp3ii2jujb/key.pub. 2024-03-24T02:51:07.456 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: The key fingerprint is: 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: SHA256:NILQoVqz8STHfZKulZpD93RSK8UKOMhxm4U2tHJk0sc ceph-2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: The key's randomart image is: 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: +---[RSA 3072]----+ 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: | ++*+. | 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: | . @*OE. . | 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: | X.%o* + + | 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: | o @ o B = . | 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: |. . o = S o | 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: | . * o + | 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: | = . | 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: | . | 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: | | 2024-03-24T02:51:07.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: +----[SHA256]-----+ 2024-03-24T02:51:07.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.896+0000 7feda57f9700 1 Processor -- start 2024-03-24T02:51:07.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.896+0000 7feda57f9700 1 -- start start 2024-03-24T02:51:07.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.896+0000 7feda57f9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 0x7feda0107f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:07.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.896+0000 7feda57f9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda01084f0 con 0x7feda0107b00 2024-03-24T02:51:07.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.896+0000 7fed9effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 0x7feda0107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.896+0000 7fed9effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 0x7feda0107f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42642/0 (socket says 172.21.15.17:42642) 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.896+0000 7fed9effd700 1 -- 172.21.15.17:0/4119033454 learned_addr learned my addr 172.21.15.17:0/4119033454 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.896+0000 7fed9effd700 1 -- 172.21.15.17:0/4119033454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda0108d00 con 0x7feda0107b00 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.897+0000 7fed9effd700 1 --2- 172.21.15.17:0/4119033454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 0x7feda0107f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fed900098a0 tx=0x7fed90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708d4c87d7037302 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.897+0000 7fed9dffb700 1 -- 172.21.15.17:0/4119033454 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed90004160 con 0x7feda0107b00 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.897+0000 7fed9dffb700 1 -- 172.21.15.17:0/4119033454 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed90004300 con 0x7feda0107b00 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.897+0000 7fed9dffb700 1 -- 172.21.15.17:0/4119033454 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed90011b50 con 0x7feda0107b00 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.897+0000 7feda57f9700 1 -- 172.21.15.17:0/4119033454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 msgr2=0x7feda0107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.897+0000 7feda57f9700 1 --2- 172.21.15.17:0/4119033454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 0x7feda0107f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fed900098a0 tx=0x7fed90009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.897+0000 7feda57f9700 1 -- 172.21.15.17:0/4119033454 shutdown_connections 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.897+0000 7feda57f9700 1 --2- 172.21.15.17:0/4119033454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 0x7feda0107f20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.897+0000 7feda57f9700 1 -- 172.21.15.17:0/4119033454 >> 172.21.15.17:0/4119033454 conn(0x7feda0103120 msgr2=0x7feda0105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:07.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.897+0000 7feda57f9700 1 -- 172.21.15.17:0/4119033454 shutdown_connections 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.897+0000 7feda57f9700 1 -- 172.21.15.17:0/4119033454 wait complete. 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.898+0000 7feda57f9700 1 Processor -- start 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.898+0000 7feda57f9700 1 -- start start 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.898+0000 7feda57f9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 0x7feda019be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.898+0000 7feda57f9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda010c0b0 con 0x7feda0107b00 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.899+0000 7fed9effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 0x7feda019be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.899+0000 7fed9effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 0x7feda019be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42650/0 (socket says 172.21.15.17:42650) 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.899+0000 7fed9effd700 1 -- 172.21.15.17:0/1782694972 learned_addr learned my addr 172.21.15.17:0/1782694972 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.899+0000 7fed9effd700 1 -- 172.21.15.17:0/1782694972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda019c360 con 0x7feda0107b00 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.899+0000 7fed9effd700 1 --2- 172.21.15.17:0/1782694972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 0x7feda019be20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fed90009fb0 tx=0x7fed90011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.899+0000 7fed87fff700 1 -- 172.21.15.17:0/1782694972 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed900038b0 con 0x7feda0107b00 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.899+0000 7fed87fff700 1 -- 172.21.15.17:0/1782694972 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed90011e50 con 0x7feda0107b00 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.899+0000 7feda57f9700 1 -- 172.21.15.17:0/1782694972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda019c5f0 con 0x7feda0107b00 2024-03-24T02:51:07.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.899+0000 7fed87fff700 1 -- 172.21.15.17:0/1782694972 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed90019550 con 0x7feda0107b00 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.899+0000 7feda57f9700 1 -- 172.21.15.17:0/1782694972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda019f2e0 con 0x7feda0107b00 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.900+0000 7fed87fff700 1 -- 172.21.15.17:0/1782694972 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fed9001a430 con 0x7feda0107b00 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.900+0000 7feda57f9700 1 -- 172.21.15.17:0/1782694972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda00637c0 con 0x7feda0107b00 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.900+0000 7fed87fff700 1 --2- 172.21.15.17:0/1782694972 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fed88038ed0 0x7fed8803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.900+0000 7fed87fff700 1 -- 172.21.15.17:0/1782694972 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fed9004d0e0 con 0x7feda0107b00 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.900+0000 7fed9e7fc700 1 --2- 172.21.15.17:0/1782694972 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fed88038ed0 0x7fed8803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.901+0000 7fed9e7fc700 1 --2- 172.21.15.17:0/1782694972 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fed88038ed0 0x7fed8803b390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed94006fd0 tx=0x7fed94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:06.905+0000 7fed87fff700 1 -- 172.21.15.17:0/1782694972 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fed90029260 con 0x7feda0107b00 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:07.112+0000 7feda57f9700 1 -- 172.21.15.17:0/1782694972 --> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7feda002daa0 con 0x7fed88038ed0 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:07.316+0000 7fed87fff700 1 -- 172.21.15.17:0/1782694972 <== mgr.14120 v2:172.21.15.17:6800/3007657427 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7feda002daa0 con 0x7fed88038ed0 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:07.318+0000 7feda57f9700 1 -- 172.21.15.17:0/1782694972 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fed88038ed0 msgr2=0x7fed8803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:07.318+0000 7feda57f9700 1 --2- 172.21.15.17:0/1782694972 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fed88038ed0 0x7fed8803b390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed94006fd0 tx=0x7fed94006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:07.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:07.318+0000 7feda57f9700 1 -- 172.21.15.17:0/1782694972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 msgr2=0x7feda019be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:07.873 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:07.318+0000 7feda57f9700 1 --2- 172.21.15.17:0/1782694972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 0x7feda019be20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fed90009fb0 tx=0x7fed90011bf0 comp rx=0 tx=0).stop 2024-03-24T02:51:07.873 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:07.318+0000 7feda57f9700 1 -- 172.21.15.17:0/1782694972 shutdown_connections 2024-03-24T02:51:07.873 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:07.318+0000 7feda57f9700 1 --2- 172.21.15.17:0/1782694972 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fed88038ed0 0x7fed8803b390 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:07.873 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:07.318+0000 7feda57f9700 1 --2- 172.21.15.17:0/1782694972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feda0107b00 0x7feda019be20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:07.873 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:07.318+0000 7feda57f9700 1 -- 172.21.15.17:0/1782694972 >> 172.21.15.17:0/1782694972 conn(0x7feda0103120 msgr2=0x7feda0103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:07.873 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:07.318+0000 7feda57f9700 1 -- 172.21.15.17:0/1782694972 shutdown_connections 2024-03-24T02:51:07.873 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:07.318+0000 7feda57f9700 1 -- 172.21.15.17:0/1782694972 wait complete. 2024-03-24T02:51:08.500 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:08 smithi017 ceph-mon[31926]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:51:08.500 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:08 smithi017 ceph-mon[31926]: Generating ssh key... 2024-03-24T02:51:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4lFKfDQCQvBQuy3vj5h7NI+H7m+BxgALhKKgys5uJU/9U5Ih0DqN6SpoTFCu7I/4pOT19nOONPdtc51M0zWRqW9oDWIOJRAw1uSgo0Z1b/Gr5uH1H3vyF8GxBCSNEz6t5FI3KWcj/8Pka7ngQHosu5u3ucWazg/FoaPfCs9tzE1xhi5OBA5g53PIE6wkaUpYIlwqQfQz9LAnayND8J6ho2yeaB3YPtUZX5OQ5Ck+fZAaMH+2QxH7ed42Ih9NYqcO4SBHMgHqcQsqzpX9D+tPdShzRohbTjS2qhg+K4uWAARO+9598ZioUwBxnUWjbPsXcC53x8tm6jEv9RoETIW/AEC/9wON5q0McTbo69fuTpbpPIWAxB6IEgql/lOn1oP70xIJpXtmJAW7csH6VxvRpzbyiY02i9TPRR2jXCwdnnobb4x3IgtXqMpL2Nn2xVlsWteCkpqAk9JUZs0mMP9J4y+Sd7pc+16HPcOnM9vIJcP3aE5eKccfhuXTRv/7S4I8= ceph-2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:51:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.284+0000 7fe99bc16700 1 Processor -- start 2024-03-24T02:51:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.284+0000 7fe99bc16700 1 -- start start 2024-03-24T02:51:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.284+0000 7fe99bc16700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 0x7fe9941067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.284+0000 7fe99bc16700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99407be30 con 0x7fe9941063d0 2024-03-24T02:51:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.284+0000 7fe9999b2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 0x7fe9941067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.284+0000 7fe9999b2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 0x7fe9941067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42652/0 (socket says 172.21.15.17:42652) 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.284+0000 7fe9999b2700 1 -- 172.21.15.17:0/1414316455 learned_addr learned my addr 172.21.15.17:0/1414316455 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.285+0000 7fe9999b2700 1 -- 172.21.15.17:0/1414316455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe994106d30 con 0x7fe9941063d0 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.285+0000 7fe9999b2700 1 --2- 172.21.15.17:0/1414316455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 0x7fe9941067f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe9900098a0 tx=0x7fe990009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8a20813a91c1f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.285+0000 7fe9989b0700 1 -- 172.21.15.17:0/1414316455 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe990004160 con 0x7fe9941063d0 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.285+0000 7fe9989b0700 1 -- 172.21.15.17:0/1414316455 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe990004300 con 0x7fe9941063d0 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.285+0000 7fe9989b0700 1 -- 172.21.15.17:0/1414316455 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe990011b50 con 0x7fe9941063d0 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.286+0000 7fe99bc16700 1 -- 172.21.15.17:0/1414316455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 msgr2=0x7fe9941067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.286+0000 7fe99bc16700 1 --2- 172.21.15.17:0/1414316455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 0x7fe9941067f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe9900098a0 tx=0x7fe990009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.286+0000 7fe99bc16700 1 -- 172.21.15.17:0/1414316455 shutdown_connections 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.286+0000 7fe99bc16700 1 --2- 172.21.15.17:0/1414316455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 0x7fe9941067f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.286+0000 7fe99bc16700 1 -- 172.21.15.17:0/1414316455 >> 172.21.15.17:0/1414316455 conn(0x7fe994102010 msgr2=0x7fe994104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.286+0000 7fe99bc16700 1 -- 172.21.15.17:0/1414316455 shutdown_connections 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.286+0000 7fe99bc16700 1 -- 172.21.15.17:0/1414316455 wait complete. 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.286+0000 7fe99bc16700 1 Processor -- start 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.286+0000 7fe99bc16700 1 -- start start 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.287+0000 7fe99bc16700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 0x7fe99419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.287+0000 7fe99bc16700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe994107dc0 con 0x7fe9941063d0 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.287+0000 7fe9999b2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 0x7fe99419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.287+0000 7fe9999b2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 0x7fe99419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42664/0 (socket says 172.21.15.17:42664) 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.287+0000 7fe9999b2700 1 -- 172.21.15.17:0/187985793 learned_addr learned my addr 172.21.15.17:0/187985793 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.287+0000 7fe9999b2700 1 -- 172.21.15.17:0/187985793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99419e500 con 0x7fe9941063d0 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.287+0000 7fe9999b2700 1 --2- 172.21.15.17:0/187985793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 0x7fe99419dfc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe990011b60 tx=0x7fe990011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.287+0000 7fe98affd700 1 -- 172.21.15.17:0/187985793 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe990003970 con 0x7fe9941063d0 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.287+0000 7fe98affd700 1 -- 172.21.15.17:0/187985793 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe990003b10 con 0x7fe9941063d0 2024-03-24T02:51:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.287+0000 7fe99bc16700 1 -- 172.21.15.17:0/187985793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99419e790 con 0x7fe9941063d0 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.287+0000 7fe98affd700 1 -- 172.21.15.17:0/187985793 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe990024610 con 0x7fe9941063d0 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.287+0000 7fe99bc16700 1 -- 172.21.15.17:0/187985793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9941a1480 con 0x7fe9941063d0 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.288+0000 7fe98affd700 1 -- 172.21.15.17:0/187985793 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fe99001a9f0 con 0x7fe9941063d0 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.288+0000 7fe99bc16700 1 -- 172.21.15.17:0/187985793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9940637c0 con 0x7fe9941063d0 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.288+0000 7fe98affd700 1 --2- 172.21.15.17:0/187985793 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fe980038f70 0x7fe98003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.288+0000 7fe98affd700 1 -- 172.21.15.17:0/187985793 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe990011e00 con 0x7fe9941063d0 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.289+0000 7fe9991b1700 1 --2- 172.21.15.17:0/187985793 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fe980038f70 0x7fe98003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.289+0000 7fe9991b1700 1 --2- 172.21.15.17:0/187985793 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fe980038f70 0x7fe98003b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe984006fd0 tx=0x7fe984006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.293+0000 7fe98affd700 1 -- 172.21.15.17:0/187985793 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe990019de0 con 0x7fe9941063d0 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.501+0000 7fe99bc16700 1 -- 172.21.15.17:0/187985793 --> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe99402daa0 con 0x7fe980038f70 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.501+0000 7fe98affd700 1 -- 172.21.15.17:0/187985793 <== mgr.14120 v2:172.21.15.17:6800/3007657427 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fe99402daa0 con 0x7fe980038f70 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.503+0000 7fe99bc16700 1 -- 172.21.15.17:0/187985793 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fe980038f70 msgr2=0x7fe98003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.503+0000 7fe99bc16700 1 --2- 172.21.15.17:0/187985793 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fe980038f70 0x7fe98003b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe984006fd0 tx=0x7fe984006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:09.063 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.503+0000 7fe99bc16700 1 -- 172.21.15.17:0/187985793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 msgr2=0x7fe99419dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:09.063 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.503+0000 7fe99bc16700 1 --2- 172.21.15.17:0/187985793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 0x7fe99419dfc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe990011b60 tx=0x7fe990011c40 comp rx=0 tx=0).stop 2024-03-24T02:51:09.063 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.504+0000 7fe99bc16700 1 -- 172.21.15.17:0/187985793 shutdown_connections 2024-03-24T02:51:09.063 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.504+0000 7fe99bc16700 1 --2- 172.21.15.17:0/187985793 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fe980038f70 0x7fe98003b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:09.063 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.504+0000 7fe99bc16700 1 --2- 172.21.15.17:0/187985793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9941063d0 0x7fe99419dfc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:09.063 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.504+0000 7fe99bc16700 1 -- 172.21.15.17:0/187985793 >> 172.21.15.17:0/187985793 conn(0x7fe994102010 msgr2=0x7fe994194980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:09.063 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.504+0000 7fe99bc16700 1 -- 172.21.15.17:0/187985793 shutdown_connections 2024-03-24T02:51:09.063 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:08.504+0000 7fe99bc16700 1 -- 172.21.15.17:0/187985793 wait complete. 2024-03-24T02:51:09.063 INFO:teuthology.orchestra.run.smithi017.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-24T02:51:09.063 INFO:teuthology.orchestra.run.smithi017.stdout:Adding key to root@localhost authorized_keys... 2024-03-24T02:51:09.063 INFO:teuthology.orchestra.run.smithi017.stdout:Adding host smithi017... 2024-03-24T02:51:09.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:09 smithi017 ceph-mon[31926]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:51:10.570 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:10 smithi017 ceph-mon[31926]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi017", "addr": "172.21.15.17", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:51:11.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:11 smithi017 ceph-mon[31926]: Deploying cephadm binary to smithi017 2024-03-24T02:51:14.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout Added host 'smithi017' with addr '172.21.15.17' 2024-03-24T02:51:14.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.500+0000 7f761a0b9700 1 Processor -- start 2024-03-24T02:51:14.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.500+0000 7f761a0b9700 1 -- start start 2024-03-24T02:51:14.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.500+0000 7f761a0b9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 0x7f7614106e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:14.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.500+0000 7f761a0b9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7614107400 con 0x7f7614106a10 2024-03-24T02:51:14.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.500+0000 7f76137fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 0x7f7614106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:14.751 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.500+0000 7f76137fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 0x7f7614106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42668/0 (socket says 172.21.15.17:42668) 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.500+0000 7f76137fe700 1 -- 172.21.15.17:0/1663003368 learned_addr learned my addr 172.21.15.17:0/1663003368 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.501+0000 7f76137fe700 1 -- 172.21.15.17:0/1663003368 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7614107c10 con 0x7f7614106a10 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.501+0000 7f76137fe700 1 --2- 172.21.15.17:0/1663003368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 0x7f7614106e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7604009e80 tx=0x7f760400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb8e32249ae69b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.501+0000 7f76127fc700 1 -- 172.21.15.17:0/1663003368 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7604017070 con 0x7f7614106a10 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.501+0000 7f76127fc700 1 -- 172.21.15.17:0/1663003368 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f760400b8c0 con 0x7f7614106a10 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.501+0000 7f76127fc700 1 -- 172.21.15.17:0/1663003368 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76040045c0 con 0x7f7614106a10 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.501+0000 7f761a0b9700 1 -- 172.21.15.17:0/1663003368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 msgr2=0x7f7614106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.501+0000 7f761a0b9700 1 --2- 172.21.15.17:0/1663003368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 0x7f7614106e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7604009e80 tx=0x7f760400b330 comp rx=0 tx=0).stop 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.502+0000 7f761a0b9700 1 -- 172.21.15.17:0/1663003368 shutdown_connections 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.502+0000 7f761a0b9700 1 --2- 172.21.15.17:0/1663003368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 0x7f7614106e30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.502+0000 7f761a0b9700 1 -- 172.21.15.17:0/1663003368 >> 172.21.15.17:0/1663003368 conn(0x7f7614102010 msgr2=0x7f7614104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.502+0000 7f761a0b9700 1 -- 172.21.15.17:0/1663003368 shutdown_connections 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.502+0000 7f761a0b9700 1 -- 172.21.15.17:0/1663003368 wait complete. 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.502+0000 7f761a0b9700 1 Processor -- start 2024-03-24T02:51:14.752 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.502+0000 7f761a0b9700 1 -- start start 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.503+0000 7f761a0b9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 0x7f761419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.503+0000 7f761a0b9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761410afc0 con 0x7f7614106a10 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.503+0000 7f76137fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 0x7f761419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.503+0000 7f76137fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 0x7f761419acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42678/0 (socket says 172.21.15.17:42678) 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.503+0000 7f76137fe700 1 -- 172.21.15.17:0/3054257431 learned_addr learned my addr 172.21.15.17:0/3054257431 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.503+0000 7f76137fe700 1 -- 172.21.15.17:0/3054257431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761419b200 con 0x7f7614106a10 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.503+0000 7f76137fe700 1 --2- 172.21.15.17:0/3054257431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 0x7f761419acc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7604004210 tx=0x7f760400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.503+0000 7f7610ff9700 1 -- 172.21.15.17:0/3054257431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7604020020 con 0x7f7614106a10 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.503+0000 7f7610ff9700 1 -- 172.21.15.17:0/3054257431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7604003b50 con 0x7f7614106a10 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.503+0000 7f761a0b9700 1 -- 172.21.15.17:0/3054257431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761419b490 con 0x7f7614106a10 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.503+0000 7f7610ff9700 1 -- 172.21.15.17:0/3054257431 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7604017040 con 0x7f7614106a10 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.503+0000 7f761a0b9700 1 -- 172.21.15.17:0/3054257431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761419e180 con 0x7f7614106a10 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.504+0000 7f7610ff9700 1 -- 172.21.15.17:0/3054257431 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f7604023020 con 0x7f7614106a10 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.504+0000 7f7610ff9700 1 --2- 172.21.15.17:0/3054257431 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f7600038f70 0x7f760003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:14.753 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.504+0000 7f7610ff9700 1 -- 172.21.15.17:0/3054257431 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f760400f050 con 0x7f7614106a10 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.504+0000 7f761a0b9700 1 -- 172.21.15.17:0/3054257431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76140637c0 con 0x7f7614106a10 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.504+0000 7f7612ffd700 1 --2- 172.21.15.17:0/3054257431 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f7600038f70 0x7f760003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.505+0000 7f7612ffd700 1 --2- 172.21.15.17:0/3054257431 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f7600038f70 0x7f760003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f75fc006fd0 tx=0x7f75fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.509+0000 7f7610ff9700 1 -- 172.21.15.17:0/3054257431 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f760402a020 con 0x7f7614106a10 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:09.719+0000 7f761a0b9700 1 -- 172.21.15.17:0/3054257431 --> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi017", "addr": "172.21.15.17", "target": ["mon-mgr", ""]}) v1 -- 0x7f761402daa0 con 0x7f7600038f70 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:14.125+0000 7f7610ff9700 1 -- 172.21.15.17:0/3054257431 <== mgr.14120 v2:172.21.15.17:6800/3007657427 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f761402daa0 con 0x7f7600038f70 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:14.127+0000 7f761a0b9700 1 -- 172.21.15.17:0/3054257431 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f7600038f70 msgr2=0x7f760003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:14.127+0000 7f761a0b9700 1 --2- 172.21.15.17:0/3054257431 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f7600038f70 0x7f760003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f75fc006fd0 tx=0x7f75fc006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:14.127+0000 7f761a0b9700 1 -- 172.21.15.17:0/3054257431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 msgr2=0x7f761419acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:14.127+0000 7f761a0b9700 1 --2- 172.21.15.17:0/3054257431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 0x7f761419acc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7604004210 tx=0x7f760400bd50 comp rx=0 tx=0).stop 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:14.127+0000 7f761a0b9700 1 -- 172.21.15.17:0/3054257431 shutdown_connections 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:14.127+0000 7f761a0b9700 1 --2- 172.21.15.17:0/3054257431 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f7600038f70 0x7f760003b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:14.754 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:14.127+0000 7f761a0b9700 1 --2- 172.21.15.17:0/3054257431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7614106a10 0x7f761419acc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:14.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:14.127+0000 7f761a0b9700 1 -- 172.21.15.17:0/3054257431 >> 172.21.15.17:0/3054257431 conn(0x7f7614102010 msgr2=0x7f7614102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:14.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:14.127+0000 7f761a0b9700 1 -- 172.21.15.17:0/3054257431 shutdown_connections 2024-03-24T02:51:14.755 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:14.127+0000 7f761a0b9700 1 -- 172.21.15.17:0/3054257431 wait complete. 2024-03-24T02:51:14.755 INFO:teuthology.orchestra.run.smithi017.stdout:Deploying unmanaged mon service... 2024-03-24T02:51:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:15 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:15 smithi017 ceph-mon[31926]: Added host smithi017 2024-03-24T02:51:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:15 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:51:16.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-24T02:51:16.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.334+0000 7f4c0acf7700 1 Processor -- start 2024-03-24T02:51:16.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.334+0000 7f4c0acf7700 1 -- start start 2024-03-24T02:51:16.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.334+0000 7f4c0acf7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 0x7f4c041061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:16.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.334+0000 7f4c0acf7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c041067c0 con 0x7f4c04105dd0 2024-03-24T02:51:16.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.334+0000 7f4c08a93700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 0x7f4c041061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:16.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.334+0000 7f4c08a93700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 0x7f4c041061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52250/0 (socket says 172.21.15.17:52250) 2024-03-24T02:51:16.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.334+0000 7f4c08a93700 1 -- 172.21.15.17:0/2441958134 learned_addr learned my addr 172.21.15.17:0/2441958134 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:16.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.334+0000 7f4c08a93700 1 -- 172.21.15.17:0/2441958134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c04108d60 con 0x7f4c04105dd0 2024-03-24T02:51:16.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.334+0000 7f4c08a93700 1 --2- 172.21.15.17:0/2441958134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 0x7f4c041061f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80098a0 tx=0x7f4bf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905732d1d838864e server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:16.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.334+0000 7f4c037fe700 1 -- 172.21.15.17:0/2441958134 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bf8004160 con 0x7f4c04105dd0 2024-03-24T02:51:16.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.335+0000 7f4c037fe700 1 -- 172.21.15.17:0/2441958134 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bf8004300 con 0x7f4c04105dd0 2024-03-24T02:51:16.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.335+0000 7f4c037fe700 1 -- 172.21.15.17:0/2441958134 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bf8011b50 con 0x7f4c04105dd0 2024-03-24T02:51:16.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.335+0000 7f4c0acf7700 1 -- 172.21.15.17:0/2441958134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 msgr2=0x7f4c041061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:16.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.335+0000 7f4c0acf7700 1 --2- 172.21.15.17:0/2441958134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 0x7f4c041061f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80098a0 tx=0x7f4bf8009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:16.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.335+0000 7f4c0acf7700 1 -- 172.21.15.17:0/2441958134 shutdown_connections 2024-03-24T02:51:16.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.335+0000 7f4c0acf7700 1 --2- 172.21.15.17:0/2441958134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 0x7f4c041061f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:16.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.335+0000 7f4c0acf7700 1 -- 172.21.15.17:0/2441958134 >> 172.21.15.17:0/2441958134 conn(0x7f4c04101740 msgr2=0x7f4c04103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:16.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.335+0000 7f4c0acf7700 1 -- 172.21.15.17:0/2441958134 shutdown_connections 2024-03-24T02:51:16.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.335+0000 7f4c0acf7700 1 -- 172.21.15.17:0/2441958134 wait complete. 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.336+0000 7f4c0acf7700 1 Processor -- start 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.336+0000 7f4c0acf7700 1 -- start start 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.336+0000 7f4c0acf7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 0x7f4c0419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.336+0000 7f4c0acf7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0410c110 con 0x7f4c04105dd0 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.336+0000 7f4c08a93700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 0x7f4c0419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.336+0000 7f4c08a93700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 0x7f4c0419bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52254/0 (socket says 172.21.15.17:52254) 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.336+0000 7f4c08a93700 1 -- 172.21.15.17:0/679020548 learned_addr learned my addr 172.21.15.17:0/679020548 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.336+0000 7f4c08a93700 1 -- 172.21.15.17:0/679020548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0419c010 con 0x7f4c04105dd0 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.336+0000 7f4c08a93700 1 --2- 172.21.15.17:0/679020548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 0x7f4c0419bad0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8011850 tx=0x7f4bf8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.337+0000 7f4c01ffb700 1 -- 172.21.15.17:0/679020548 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bf8003af0 con 0x7f4c04105dd0 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.337+0000 7f4c01ffb700 1 -- 172.21.15.17:0/679020548 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bf8020070 con 0x7f4c04105dd0 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.337+0000 7f4c0acf7700 1 -- 172.21.15.17:0/679020548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0419c2a0 con 0x7f4c04105dd0 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.337+0000 7f4c01ffb700 1 -- 172.21.15.17:0/679020548 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bf80195a0 con 0x7f4c04105dd0 2024-03-24T02:51:16.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.337+0000 7f4c0acf7700 1 -- 172.21.15.17:0/679020548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0419ef90 con 0x7f4c04105dd0 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.338+0000 7f4c01ffb700 1 -- 172.21.15.17:0/679020548 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f4bf801a430 con 0x7f4c04105dd0 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.338+0000 7f4c0acf7700 1 -- 172.21.15.17:0/679020548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c040637c0 con 0x7f4c04105dd0 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.338+0000 7f4c01ffb700 1 --2- 172.21.15.17:0/679020548 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f4bf4038f70 0x7f4bf403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.338+0000 7f4c01ffb700 1 -- 172.21.15.17:0/679020548 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4bf8011e00 con 0x7f4c04105dd0 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.338+0000 7f4c03fff700 1 --2- 172.21.15.17:0/679020548 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f4bf4038f70 0x7f4bf403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.339+0000 7f4c03fff700 1 --2- 172.21.15.17:0/679020548 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f4bf4038f70 0x7f4bf403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0006fd0 tx=0x7f4bf0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.343+0000 7f4c01ffb700 1 -- 172.21.15.17:0/679020548 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4bf8021270 con 0x7f4c04105dd0 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.563+0000 7f4c0acf7700 1 -- 172.21.15.17:0/679020548 --> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4c04102910 con 0x7f4bf4038f70 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.568+0000 7f4c01ffb700 1 -- 172.21.15.17:0/679020548 <== mgr.14120 v2:172.21.15.17:6800/3007657427 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4c04102910 con 0x7f4bf4038f70 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.570+0000 7f4c0acf7700 1 -- 172.21.15.17:0/679020548 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f4bf4038f70 msgr2=0x7f4bf403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.570+0000 7f4c0acf7700 1 --2- 172.21.15.17:0/679020548 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f4bf4038f70 0x7f4bf403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0006fd0 tx=0x7f4bf0006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.570+0000 7f4c0acf7700 1 -- 172.21.15.17:0/679020548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 msgr2=0x7f4c0419bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.570+0000 7f4c0acf7700 1 --2- 172.21.15.17:0/679020548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 0x7f4c0419bad0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8011850 tx=0x7f4bf8011c10 comp rx=0 tx=0).stop 2024-03-24T02:51:16.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.570+0000 7f4c0acf7700 1 -- 172.21.15.17:0/679020548 shutdown_connections 2024-03-24T02:51:16.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.570+0000 7f4c0acf7700 1 --2- 172.21.15.17:0/679020548 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f4bf4038f70 0x7f4bf403b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:16.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.570+0000 7f4c0acf7700 1 --2- 172.21.15.17:0/679020548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c04105dd0 0x7f4c0419bad0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:16.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.570+0000 7f4c0acf7700 1 -- 172.21.15.17:0/679020548 >> 172.21.15.17:0/679020548 conn(0x7f4c04101740 msgr2=0x7f4c04102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:16.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.570+0000 7f4c0acf7700 1 -- 172.21.15.17:0/679020548 shutdown_connections 2024-03-24T02:51:16.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:15.570+0000 7f4c0acf7700 1 -- 172.21.15.17:0/679020548 wait complete. 2024-03-24T02:51:16.105 INFO:teuthology.orchestra.run.smithi017.stdout:Deploying unmanaged mgr service... 2024-03-24T02:51:16.728 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:16 smithi017 ceph-mon[31926]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:51:16.728 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:16 smithi017 ceph-mon[31926]: Saving service mon spec with placement count:5 2024-03-24T02:51:16.728 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:16 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:17.320 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-24T02:51:17.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.499+0000 7f0f3b2f3700 1 Processor -- start 2024-03-24T02:51:17.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.499+0000 7f0f3b2f3700 1 -- start start 2024-03-24T02:51:17.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.499+0000 7f0f3b2f3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 0x7f0f3407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:17.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.499+0000 7f0f3b2f3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3407c890 con 0x7f0f3407de60 2024-03-24T02:51:17.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.500+0000 7f0f3908f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 0x7f0f3407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:17.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.500+0000 7f0f3908f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 0x7f0f3407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52266/0 (socket says 172.21.15.17:52266) 2024-03-24T02:51:17.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.500+0000 7f0f3908f700 1 -- 172.21.15.17:0/4133499986 learned_addr learned my addr 172.21.15.17:0/4133499986 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.500+0000 7f0f3908f700 1 -- 172.21.15.17:0/4133499986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3407ca10 con 0x7f0f3407de60 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.500+0000 7f0f3908f700 1 --2- 172.21.15.17:0/4133499986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 0x7f0f3407c2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f300098a0 tx=0x7f0f30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c82c19a06e02030 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.500+0000 7f0f2bfff700 1 -- 172.21.15.17:0/4133499986 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f30004160 con 0x7f0f3407de60 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.500+0000 7f0f2bfff700 1 -- 172.21.15.17:0/4133499986 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f30004300 con 0x7f0f3407de60 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.500+0000 7f0f2bfff700 1 -- 172.21.15.17:0/4133499986 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f30011b50 con 0x7f0f3407de60 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.501+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/4133499986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 msgr2=0x7f0f3407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.501+0000 7f0f3b2f3700 1 --2- 172.21.15.17:0/4133499986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 0x7f0f3407c2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f300098a0 tx=0x7f0f30009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.501+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/4133499986 shutdown_connections 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.501+0000 7f0f3b2f3700 1 --2- 172.21.15.17:0/4133499986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 0x7f0f3407c2c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.501+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/4133499986 >> 172.21.15.17:0/4133499986 conn(0x7f0f341030b0 msgr2=0x7f0f341054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.501+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/4133499986 shutdown_connections 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.501+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/4133499986 wait complete. 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f3b2f3700 1 Processor -- start 2024-03-24T02:51:17.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f3b2f3700 1 -- start start 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f3b2f3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 0x7f0f3419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f3b2f3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3410e120 con 0x7f0f3407de60 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f3908f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 0x7f0f3419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f3908f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 0x7f0f3419e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52270/0 (socket says 172.21.15.17:52270) 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f3908f700 1 -- 172.21.15.17:0/3012998569 learned_addr learned my addr 172.21.15.17:0/3012998569 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f3908f700 1 -- 172.21.15.17:0/3012998569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3419e560 con 0x7f0f3407de60 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f3908f700 1 --2- 172.21.15.17:0/3012998569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 0x7f0f3419e020 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009fb0 tx=0x7f0f30011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f2a7fc700 1 -- 172.21.15.17:0/3012998569 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f300038b0 con 0x7f0f3407de60 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f2a7fc700 1 -- 172.21.15.17:0/3012998569 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f30011e50 con 0x7f0f3407de60 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/3012998569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3419e7f0 con 0x7f0f3407de60 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f2a7fc700 1 -- 172.21.15.17:0/3012998569 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f30019420 con 0x7f0f3407de60 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.502+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/3012998569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f341a14e0 con 0x7f0f3407de60 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.503+0000 7f0f2a7fc700 1 -- 172.21.15.17:0/3012998569 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f0f30003a50 con 0x7f0f3407de60 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.503+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/3012998569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f340637c0 con 0x7f0f3407de60 2024-03-24T02:51:17.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.503+0000 7f0f2a7fc700 1 --2- 172.21.15.17:0/3012998569 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f0f20038f20 0x7f0f2003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.504+0000 7f0f2a7fc700 1 -- 172.21.15.17:0/3012998569 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0f3004d060 con 0x7f0f3407de60 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.504+0000 7f0f3888e700 1 --2- 172.21.15.17:0/3012998569 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f0f20038f20 0x7f0f2003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.504+0000 7f0f3888e700 1 --2- 172.21.15.17:0/3012998569 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f0f20038f20 0x7f0f2003b3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0f24006fd0 tx=0x7f0f24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.508+0000 7f0f2a7fc700 1 -- 172.21.15.17:0/3012998569 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0f30023390 con 0x7f0f3407de60 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.729+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/3012998569 --> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0f3402daa0 con 0x7f0f20038f20 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.733+0000 7f0f2a7fc700 1 -- 172.21.15.17:0/3012998569 <== mgr.14120 v2:172.21.15.17:6800/3007657427 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0f3402daa0 con 0x7f0f20038f20 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.735+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/3012998569 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f0f20038f20 msgr2=0x7f0f2003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.735+0000 7f0f3b2f3700 1 --2- 172.21.15.17:0/3012998569 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f0f20038f20 0x7f0f2003b3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0f24006fd0 tx=0x7f0f24006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.735+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/3012998569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 msgr2=0x7f0f3419e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.735+0000 7f0f3b2f3700 1 --2- 172.21.15.17:0/3012998569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 0x7f0f3419e020 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009fb0 tx=0x7f0f30011bf0 comp rx=0 tx=0).stop 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.735+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/3012998569 shutdown_connections 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.735+0000 7f0f3b2f3700 1 --2- 172.21.15.17:0/3012998569 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f0f20038f20 0x7f0f2003b3e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.735+0000 7f0f3b2f3700 1 --2- 172.21.15.17:0/3012998569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f3407de60 0x7f0f3419e020 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.735+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/3012998569 >> 172.21.15.17:0/3012998569 conn(0x7f0f341030b0 msgr2=0x7f0f34103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.735+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/3012998569 shutdown_connections 2024-03-24T02:51:17.330 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:16.735+0000 7f0f3b2f3700 1 -- 172.21.15.17:0/3012998569 wait complete. 2024-03-24T02:51:17.930 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:17 smithi017 ceph-mon[31926]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:51:17.930 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:17 smithi017 ceph-mon[31926]: Saving service mgr spec with placement count:2 2024-03-24T02:51:17.930 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:17 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:19.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.958+0000 7f155337b700 1 Processor -- start 2024-03-24T02:51:19.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.958+0000 7f155337b700 1 -- start start 2024-03-24T02:51:19.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.958+0000 7f155337b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 0x7f154c107f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:19.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.958+0000 7f155337b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c1084d0 con 0x7f154c107ae0 2024-03-24T02:51:19.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.958+0000 7f1551117700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 0x7f154c107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:19.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.959+0000 7f1551117700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 0x7f154c107f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52280/0 (socket says 172.21.15.17:52280) 2024-03-24T02:51:19.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.959+0000 7f1551117700 1 -- 172.21.15.17:0/211877451 learned_addr learned my addr 172.21.15.17:0/211877451 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:19.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.959+0000 7f1551117700 1 -- 172.21.15.17:0/211877451 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c108ce0 con 0x7f154c107ae0 2024-03-24T02:51:19.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.959+0000 7f1551117700 1 --2- 172.21.15.17:0/211877451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 0x7f154c107f00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f15440098a0 tx=0x7f1544009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9c07cee91b888f server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:19.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.959+0000 7f154bfff700 1 -- 172.21.15.17:0/211877451 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1544004160 con 0x7f154c107ae0 2024-03-24T02:51:19.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.959+0000 7f154bfff700 1 -- 172.21.15.17:0/211877451 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1544004300 con 0x7f154c107ae0 2024-03-24T02:51:19.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.959+0000 7f154bfff700 1 -- 172.21.15.17:0/211877451 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1544011b50 con 0x7f154c107ae0 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.960+0000 7f155337b700 1 -- 172.21.15.17:0/211877451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 msgr2=0x7f154c107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.960+0000 7f155337b700 1 --2- 172.21.15.17:0/211877451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 0x7f154c107f00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f15440098a0 tx=0x7f1544009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.960+0000 7f155337b700 1 -- 172.21.15.17:0/211877451 shutdown_connections 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.960+0000 7f155337b700 1 --2- 172.21.15.17:0/211877451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 0x7f154c107f00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.960+0000 7f155337b700 1 -- 172.21.15.17:0/211877451 >> 172.21.15.17:0/211877451 conn(0x7f154c103080 msgr2=0x7f154c1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.960+0000 7f155337b700 1 -- 172.21.15.17:0/211877451 shutdown_connections 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.960+0000 7f155337b700 1 -- 172.21.15.17:0/211877451 wait complete. 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.960+0000 7f155337b700 1 Processor -- start 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.960+0000 7f155337b700 1 -- start start 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.961+0000 7f155337b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 0x7f154c07db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.961+0000 7f155337b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c10c090 con 0x7f154c107ae0 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.961+0000 7f1551117700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 0x7f154c07db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.961+0000 7f1551117700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 0x7f154c07db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52286/0 (socket says 172.21.15.17:52286) 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.961+0000 7f1551117700 1 -- 172.21.15.17:0/3101604448 learned_addr learned my addr 172.21.15.17:0/3101604448 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.961+0000 7f1551117700 1 -- 172.21.15.17:0/3101604448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c07e090 con 0x7f154c107ae0 2024-03-24T02:51:19.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.961+0000 7f1551117700 1 --2- 172.21.15.17:0/3101604448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 0x7f154c07db50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1544011850 tx=0x7f1544011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.961+0000 7f154a7fc700 1 -- 172.21.15.17:0/3101604448 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1544003af0 con 0x7f154c107ae0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.961+0000 7f155337b700 1 -- 172.21.15.17:0/3101604448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c07c280 con 0x7f154c107ae0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.961+0000 7f154a7fc700 1 -- 172.21.15.17:0/3101604448 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1544020070 con 0x7f154c107ae0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.961+0000 7f154a7fc700 1 -- 172.21.15.17:0/3101604448 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15440195a0 con 0x7f154c107ae0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.961+0000 7f155337b700 1 -- 172.21.15.17:0/3101604448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c07c7c0 con 0x7f154c107ae0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.962+0000 7f154a7fc700 1 -- 172.21.15.17:0/3101604448 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f1544019850 con 0x7f154c107ae0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.962+0000 7f155337b700 1 -- 172.21.15.17:0/3101604448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c0637c0 con 0x7f154c107ae0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.962+0000 7f154a7fc700 1 --2- 172.21.15.17:0/3101604448 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f1534038f70 0x7f153403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.962+0000 7f154a7fc700 1 -- 172.21.15.17:0/3101604448 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1544011e00 con 0x7f154c107ae0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.962+0000 7f1550916700 1 --2- 172.21.15.17:0/3101604448 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f1534038f70 0x7f153403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.963+0000 7f1550916700 1 --2- 172.21.15.17:0/3101604448 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f1534038f70 0x7f153403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1540006fd0 tx=0x7f1540006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:17.967+0000 7f154a7fc700 1 -- 172.21.15.17:0/3101604448 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1544019b40 con 0x7f154c107ae0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:18.171+0000 7f155337b700 1 -- 172.21.15.17:0/3101604448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f154c195560 con 0x7f154c107ae0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:18.173+0000 7f154a7fc700 1 -- 172.21.15.17:0/3101604448 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) v1 ==== 142+0+0 (secure 0 0 0) 0x7f1544003c90 con 0x7f154c107ae0 2024-03-24T02:51:19.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:18.175+0000 7f155337b700 1 -- 172.21.15.17:0/3101604448 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f1534038f70 msgr2=0x7f153403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:19.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:18.175+0000 7f155337b700 1 --2- 172.21.15.17:0/3101604448 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f1534038f70 0x7f153403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1540006fd0 tx=0x7f1540006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:19.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:18.175+0000 7f155337b700 1 -- 172.21.15.17:0/3101604448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 msgr2=0x7f154c07db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:19.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:18.175+0000 7f155337b700 1 --2- 172.21.15.17:0/3101604448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 0x7f154c07db50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1544011850 tx=0x7f1544011c10 comp rx=0 tx=0).stop 2024-03-24T02:51:19.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:18.175+0000 7f155337b700 1 -- 172.21.15.17:0/3101604448 shutdown_connections 2024-03-24T02:51:19.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:18.175+0000 7f155337b700 1 --2- 172.21.15.17:0/3101604448 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f1534038f70 0x7f153403b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:19.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:18.175+0000 7f155337b700 1 --2- 172.21.15.17:0/3101604448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f154c107ae0 0x7f154c07db50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:19.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:18.175+0000 7f155337b700 1 -- 172.21.15.17:0/3101604448 >> 172.21.15.17:0/3101604448 conn(0x7f154c103080 msgr2=0x7f154c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:19.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:18.176+0000 7f155337b700 1 -- 172.21.15.17:0/3101604448 shutdown_connections 2024-03-24T02:51:19.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:18.176+0000 7f155337b700 1 -- 172.21.15.17:0/3101604448 wait complete. 2024-03-24T02:51:19.435 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:19 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3101604448' entity='client.admin' 2024-03-24T02:51:19.435 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:19 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:20.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:20 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:20.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:20 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1964067231' entity='client.admin' 2024-03-24T02:51:21.115 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.645+0000 7fab24665700 1 Processor -- start 2024-03-24T02:51:21.115 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.645+0000 7fab24665700 1 -- start start 2024-03-24T02:51:21.115 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.646+0000 7fab24665700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 0x7fab1c107f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.646+0000 7fab24665700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c108500 con 0x7fab1c107b10 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.646+0000 7fab22401700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 0x7fab1c107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.646+0000 7fab22401700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 0x7fab1c107f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52298/0 (socket says 172.21.15.17:52298) 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.646+0000 7fab22401700 1 -- 172.21.15.17:0/599942358 learned_addr learned my addr 172.21.15.17:0/599942358 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.647+0000 7fab22401700 1 -- 172.21.15.17:0/599942358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c108d10 con 0x7fab1c107b10 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.647+0000 7fab22401700 1 --2- 172.21.15.17:0/599942358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 0x7fab1c107f30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab180098a0 tx=0x7fab18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffb628542172a53 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.647+0000 7fab213ff700 1 -- 172.21.15.17:0/599942358 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab18004160 con 0x7fab1c107b10 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.647+0000 7fab213ff700 1 -- 172.21.15.17:0/599942358 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab18004300 con 0x7fab1c107b10 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.647+0000 7fab213ff700 1 -- 172.21.15.17:0/599942358 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab18011b20 con 0x7fab1c107b10 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.648+0000 7fab24665700 1 -- 172.21.15.17:0/599942358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 msgr2=0x7fab1c107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.648+0000 7fab24665700 1 --2- 172.21.15.17:0/599942358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 0x7fab1c107f30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab180098a0 tx=0x7fab18009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.648+0000 7fab24665700 1 -- 172.21.15.17:0/599942358 shutdown_connections 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.648+0000 7fab24665700 1 --2- 172.21.15.17:0/599942358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 0x7fab1c107f30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:21.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.648+0000 7fab24665700 1 -- 172.21.15.17:0/599942358 >> 172.21.15.17:0/599942358 conn(0x7fab1c1030d0 msgr2=0x7fab1c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.648+0000 7fab24665700 1 -- 172.21.15.17:0/599942358 shutdown_connections 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.648+0000 7fab24665700 1 -- 172.21.15.17:0/599942358 wait complete. 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.648+0000 7fab24665700 1 Processor -- start 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.649+0000 7fab24665700 1 -- start start 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.649+0000 7fab24665700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 0x7fab1c19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.649+0000 7fab24665700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c10c0c0 con 0x7fab1c107b10 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.649+0000 7fab22401700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 0x7fab1c19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.649+0000 7fab22401700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 0x7fab1c19be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52306/0 (socket says 172.21.15.17:52306) 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.649+0000 7fab22401700 1 -- 172.21.15.17:0/1964067231 learned_addr learned my addr 172.21.15.17:0/1964067231 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.649+0000 7fab22401700 1 -- 172.21.15.17:0/1964067231 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c19c340 con 0x7fab1c107b10 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.649+0000 7fab22401700 1 --2- 172.21.15.17:0/1964067231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 0x7fab1c19be00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab180116a0 tx=0x7fab18011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.649+0000 7fab137fe700 1 -- 172.21.15.17:0/1964067231 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab18003a40 con 0x7fab1c107b10 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.649+0000 7fab137fe700 1 -- 172.21.15.17:0/1964067231 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab18003be0 con 0x7fab1c107b10 2024-03-24T02:51:21.117 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.649+0000 7fab24665700 1 -- 172.21.15.17:0/1964067231 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c19c5d0 con 0x7fab1c107b10 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.649+0000 7fab137fe700 1 -- 172.21.15.17:0/1964067231 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab1801a720 con 0x7fab1c107b10 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.650+0000 7fab24665700 1 -- 172.21.15.17:0/1964067231 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c19ca30 con 0x7fab1c107b10 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.650+0000 7fab137fe700 1 -- 172.21.15.17:0/1964067231 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fab1801a9d0 con 0x7fab1c107b10 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.650+0000 7fab24665700 1 -- 172.21.15.17:0/1964067231 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c0637c0 con 0x7fab1c107b10 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.650+0000 7fab137fe700 1 --2- 172.21.15.17:0/1964067231 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fab08038f20 0x7fab0803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.651+0000 7fab137fe700 1 -- 172.21.15.17:0/1964067231 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fab1804e350 con 0x7fab1c107b10 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.651+0000 7fab21c00700 1 --2- 172.21.15.17:0/1964067231 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fab08038f20 0x7fab0803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.651+0000 7fab21c00700 1 --2- 172.21.15.17:0/1964067231 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fab08038f20 0x7fab0803b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fab0c006fd0 tx=0x7fab0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.655+0000 7fab137fe700 1 -- 172.21.15.17:0/1964067231 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab18011e20 con 0x7fab1c107b10 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.868+0000 7fab24665700 1 -- 172.21.15.17:0/1964067231 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fab1c02db40 con 0x7fab1c107b10 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.871+0000 7fab137fe700 1 -- 172.21.15.17:0/1964067231 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fab18019c70 con 0x7fab1c107b10 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.873+0000 7fab24665700 1 -- 172.21.15.17:0/1964067231 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fab08038f20 msgr2=0x7fab0803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:21.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.873+0000 7fab24665700 1 --2- 172.21.15.17:0/1964067231 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fab08038f20 0x7fab0803b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fab0c006fd0 tx=0x7fab0c006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:21.119 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.873+0000 7fab24665700 1 -- 172.21.15.17:0/1964067231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 msgr2=0x7fab1c19be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:21.119 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.873+0000 7fab24665700 1 --2- 172.21.15.17:0/1964067231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 0x7fab1c19be00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab180116a0 tx=0x7fab18011ba0 comp rx=0 tx=0).stop 2024-03-24T02:51:21.119 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.873+0000 7fab24665700 1 -- 172.21.15.17:0/1964067231 shutdown_connections 2024-03-24T02:51:21.119 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.873+0000 7fab24665700 1 --2- 172.21.15.17:0/1964067231 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7fab08038f20 0x7fab0803b3e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:21.119 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.873+0000 7fab24665700 1 --2- 172.21.15.17:0/1964067231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab1c107b10 0x7fab1c19be00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:21.119 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.873+0000 7fab24665700 1 -- 172.21.15.17:0/1964067231 >> 172.21.15.17:0/1964067231 conn(0x7fab1c1030d0 msgr2=0x7fab1c103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:21.119 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.874+0000 7fab24665700 1 -- 172.21.15.17:0/1964067231 shutdown_connections 2024-03-24T02:51:21.119 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:19.874+0000 7fab24665700 1 -- 172.21.15.17:0/1964067231 wait complete. 2024-03-24T02:51:21.119 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling the dashboard module... 2024-03-24T02:51:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:21 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:22.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:22 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2436097490' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-24T02:51:22.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:22 smithi017 ceph-mon[31926]: from='mgr.14120 172.21.15.17:0/541739172' entity='mgr.x' 2024-03-24T02:51:22.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:22 smithi017 ceph-mon[31926]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:51:22.957 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:22 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: ignoring --setuser ceph since I am not root 2024-03-24T02:51:22.957 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:22 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: ignoring --setgroup ceph since I am not root 2024-03-24T02:51:23.597 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:23 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:23.396+0000 7f5f594fd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T02:51:23.597 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:23 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:23.491+0000 7f5f594fd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T02:51:23.916 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:23 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2436097490' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-24T02:51:23.916 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:23 smithi017 ceph-mon[31926]: mgrmap e9: x(active, since 20s) 2024-03-24T02:51:24.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.573+0000 7f46ce57e700 1 Processor -- start 2024-03-24T02:51:24.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.573+0000 7f46ce57e700 1 -- start start 2024-03-24T02:51:24.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.573+0000 7f46ce57e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 0x7f46c81067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:24.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.573+0000 7f46ce57e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c807be30 con 0x7f46c81063a0 2024-03-24T02:51:24.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.573+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 0x7f46c81067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:24.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.573+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 0x7f46c81067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45118/0 (socket says 172.21.15.17:45118) 2024-03-24T02:51:24.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.573+0000 7f46c7fff700 1 -- 172.21.15.17:0/1974450756 learned_addr learned my addr 172.21.15.17:0/1974450756 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:24.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.574+0000 7f46c7fff700 1 -- 172.21.15.17:0/1974450756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c8106d00 con 0x7f46c81063a0 2024-03-24T02:51:24.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.574+0000 7f46c7fff700 1 --2- 172.21.15.17:0/1974450756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 0x7f46c81067c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f46b80098a0 tx=0x7f46b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d702cf7016f9b369 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:24.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.574+0000 7f46c6ffd700 1 -- 172.21.15.17:0/1974450756 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46b8004160 con 0x7f46c81063a0 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.574+0000 7f46c6ffd700 1 -- 172.21.15.17:0/1974450756 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46b8004300 con 0x7f46c81063a0 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.574+0000 7f46c6ffd700 1 -- 172.21.15.17:0/1974450756 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46b8011b20 con 0x7f46c81063a0 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.575+0000 7f46ce57e700 1 -- 172.21.15.17:0/1974450756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 msgr2=0x7f46c81067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.575+0000 7f46ce57e700 1 --2- 172.21.15.17:0/1974450756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 0x7f46c81067c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f46b80098a0 tx=0x7f46b8009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.575+0000 7f46ce57e700 1 -- 172.21.15.17:0/1974450756 shutdown_connections 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.575+0000 7f46ce57e700 1 --2- 172.21.15.17:0/1974450756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 0x7f46c81067c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.575+0000 7f46ce57e700 1 -- 172.21.15.17:0/1974450756 >> 172.21.15.17:0/1974450756 conn(0x7f46c8102000 msgr2=0x7f46c8104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.575+0000 7f46ce57e700 1 -- 172.21.15.17:0/1974450756 shutdown_connections 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.575+0000 7f46ce57e700 1 -- 172.21.15.17:0/1974450756 wait complete. 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.575+0000 7f46ce57e700 1 Processor -- start 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.575+0000 7f46ce57e700 1 -- start start 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.576+0000 7f46ce57e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 0x7f46c819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:24.159 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.576+0000 7f46ce57e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8107dd0 con 0x7f46c81063a0 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.576+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 0x7f46c819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.576+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 0x7f46c819dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45132/0 (socket says 172.21.15.17:45132) 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.576+0000 7f46c7fff700 1 -- 172.21.15.17:0/2436097490 learned_addr learned my addr 172.21.15.17:0/2436097490 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.576+0000 7f46c7fff700 1 -- 172.21.15.17:0/2436097490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c819e1a0 con 0x7f46c81063a0 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.576+0000 7f46c7fff700 1 --2- 172.21.15.17:0/2436097490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 0x7f46c819dc60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f46b80116a0 tx=0x7f46b8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.576+0000 7f46c57fa700 1 -- 172.21.15.17:0/2436097490 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46b8003a40 con 0x7f46c81063a0 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.576+0000 7f46c57fa700 1 -- 172.21.15.17:0/2436097490 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46b8003be0 con 0x7f46c81063a0 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.576+0000 7f46ce57e700 1 -- 172.21.15.17:0/2436097490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c819e430 con 0x7f46c81063a0 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.576+0000 7f46c57fa700 1 -- 172.21.15.17:0/2436097490 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46b801a750 con 0x7f46c81063a0 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.576+0000 7f46ce57e700 1 -- 172.21.15.17:0/2436097490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c819e890 con 0x7f46c81063a0 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.577+0000 7f46c57fa700 1 -- 172.21.15.17:0/2436097490 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f46b801aa80 con 0x7f46c81063a0 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.577+0000 7f46ce57e700 1 -- 172.21.15.17:0/2436097490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c80637c0 con 0x7f46c81063a0 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.577+0000 7f46c57fa700 1 --2- 172.21.15.17:0/2436097490 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f46b0038f20 0x7f46b003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.578+0000 7f46c57fa700 1 -- 172.21.15.17:0/2436097490 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f46b804e310 con 0x7f46c81063a0 2024-03-24T02:51:24.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.578+0000 7f46c77fe700 1 --2- 172.21.15.17:0/2436097490 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f46b0038f20 0x7f46b003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.578+0000 7f46c77fe700 1 --2- 172.21.15.17:0/2436097490 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f46b0038f20 0x7f46b003b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006fd0 tx=0x7f46bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.583+0000 7f46c57fa700 1 -- 172.21.15.17:0/2436097490 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f46b8011e20 con 0x7f46c81063a0 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:21.848+0000 7f46ce57e700 1 -- 172.21.15.17:0/2436097490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f46c8062da0 con 0x7f46c81063a0 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:22.816+0000 7f46c57fa700 1 -- 172.21.15.17:0/2436097490 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f46b800bc70 con 0x7f46c81063a0 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:22.818+0000 7f46c57fa700 1 -- 172.21.15.17:0/2436097490 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f46b802d910 con 0x7f46c81063a0 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:22.820+0000 7f46ce57e700 1 -- 172.21.15.17:0/2436097490 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f46b0038f20 msgr2=0x7f46b003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:22.820+0000 7f46ce57e700 1 --2- 172.21.15.17:0/2436097490 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f46b0038f20 0x7f46b003b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006fd0 tx=0x7f46bc006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:22.820+0000 7f46ce57e700 1 -- 172.21.15.17:0/2436097490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 msgr2=0x7f46c819dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:22.820+0000 7f46ce57e700 1 --2- 172.21.15.17:0/2436097490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 0x7f46c819dc60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f46b80116a0 tx=0x7f46b8011ba0 comp rx=0 tx=0).stop 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:22.820+0000 7f46ce57e700 1 -- 172.21.15.17:0/2436097490 shutdown_connections 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:22.820+0000 7f46ce57e700 1 --2- 172.21.15.17:0/2436097490 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f46b0038f20 0x7f46b003b3e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:22.820+0000 7f46ce57e700 1 --2- 172.21.15.17:0/2436097490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46c81063a0 0x7f46c819dc60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:22.820+0000 7f46ce57e700 1 -- 172.21.15.17:0/2436097490 >> 172.21.15.17:0/2436097490 conn(0x7f46c8102000 msgr2=0x7f46c8194900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:22.820+0000 7f46ce57e700 1 -- 172.21.15.17:0/2436097490 shutdown_connections 2024-03-24T02:51:24.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:22.820+0000 7f46ce57e700 1 -- 172.21.15.17:0/2436097490 wait complete. 2024-03-24T02:51:24.205 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:24 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:24.149+0000 7f5f594fd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T02:51:24.512 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:24 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:24.247+0000 7f5f594fd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T02:51:24.513 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:24 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:24.393+0000 7f5f594fd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T02:51:24.513 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:24 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:24.492+0000 7f5f594fd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T02:51:24.818 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:24 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:24.580+0000 7f5f594fd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T02:51:25.161 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:24 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1902968643' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-24T02:51:25.161 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:24 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:24.854+0000 7f5f594fd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T02:51:25.161 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:24 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:24.970+0000 7f5f594fd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T02:51:25.161 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:25 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:25.059+0000 7f5f594fd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T02:51:25.456 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:25 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:25.161+0000 7f5f594fd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T02:51:25.456 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:25 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:25.340+0000 7f5f594fd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T02:51:25.507 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-03-24T02:51:25.507 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-24T02:51:25.507 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-24T02:51:25.507 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-24T02:51:25.507 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-24T02:51:25.507 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-03-24T02:51:25.507 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.613+0000 7f6673d2e700 1 Processor -- start 2024-03-24T02:51:25.507 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.613+0000 7f6673d2e700 1 -- start start 2024-03-24T02:51:25.507 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.614+0000 7f6673d2e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 0x7f666c107f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:25.507 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.614+0000 7f6673d2e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c1084e0 con 0x7f666c107af0 2024-03-24T02:51:25.507 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.614+0000 7f6671aca700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 0x7f666c107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:25.507 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.614+0000 7f6671aca700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 0x7f666c107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45166/0 (socket says 172.21.15.17:45166) 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.614+0000 7f6671aca700 1 -- 172.21.15.17:0/4130212524 learned_addr learned my addr 172.21.15.17:0/4130212524 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.614+0000 7f6671aca700 1 -- 172.21.15.17:0/4130212524 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c108cf0 con 0x7f666c107af0 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.614+0000 7f6671aca700 1 --2- 172.21.15.17:0/4130212524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 0x7f666c107f10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66680098a0 tx=0x7f6668009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c60ae800d3f45f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.615+0000 7f6670ac8700 1 -- 172.21.15.17:0/4130212524 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6668004160 con 0x7f666c107af0 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.615+0000 7f6670ac8700 1 -- 172.21.15.17:0/4130212524 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6668004300 con 0x7f666c107af0 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.615+0000 7f6670ac8700 1 -- 172.21.15.17:0/4130212524 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6668011b20 con 0x7f666c107af0 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.615+0000 7f6673d2e700 1 -- 172.21.15.17:0/4130212524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 msgr2=0x7f666c107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.615+0000 7f6673d2e700 1 --2- 172.21.15.17:0/4130212524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 0x7f666c107f10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66680098a0 tx=0x7f6668009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.615+0000 7f6673d2e700 1 -- 172.21.15.17:0/4130212524 shutdown_connections 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.615+0000 7f6673d2e700 1 --2- 172.21.15.17:0/4130212524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 0x7f666c107f10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.615+0000 7f6673d2e700 1 -- 172.21.15.17:0/4130212524 >> 172.21.15.17:0/4130212524 conn(0x7f666c1030b0 msgr2=0x7f666c1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.615+0000 7f6673d2e700 1 -- 172.21.15.17:0/4130212524 shutdown_connections 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.615+0000 7f6673d2e700 1 -- 172.21.15.17:0/4130212524 wait complete. 2024-03-24T02:51:25.508 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.616+0000 7f6673d2e700 1 Processor -- start 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.616+0000 7f6673d2e700 1 -- start start 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.616+0000 7f6673d2e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 0x7f666c19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.616+0000 7f6673d2e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c10c0a0 con 0x7f666c107af0 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.616+0000 7f6671aca700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 0x7f666c19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.616+0000 7f6671aca700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 0x7f666c19bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45182/0 (socket says 172.21.15.17:45182) 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.616+0000 7f6671aca700 1 -- 172.21.15.17:0/1902968643 learned_addr learned my addr 172.21.15.17:0/1902968643 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.616+0000 7f6671aca700 1 -- 172.21.15.17:0/1902968643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c19c330 con 0x7f666c107af0 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.617+0000 7f6671aca700 1 --2- 172.21.15.17:0/1902968643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 0x7f666c19bdf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66680116a0 tx=0x7f6668011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.617+0000 7f665effd700 1 -- 172.21.15.17:0/1902968643 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6668003a40 con 0x7f666c107af0 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.617+0000 7f665effd700 1 -- 172.21.15.17:0/1902968643 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6668003be0 con 0x7f666c107af0 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.617+0000 7f6673d2e700 1 -- 172.21.15.17:0/1902968643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666c19c5c0 con 0x7f666c107af0 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.617+0000 7f665effd700 1 -- 172.21.15.17:0/1902968643 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f666801a720 con 0x7f666c107af0 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.617+0000 7f6673d2e700 1 -- 172.21.15.17:0/1902968643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666c19ca20 con 0x7f666c107af0 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.618+0000 7f6673d2e700 1 -- 172.21.15.17:0/1902968643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666c0637c0 con 0x7f666c107af0 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.618+0000 7f665effd700 1 -- 172.21.15.17:0/1902968643 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f666801aa50 con 0x7f666c107af0 2024-03-24T02:51:25.509 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.618+0000 7f665effd700 1 --2- 172.21.15.17:0/1902968643 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f6658038f80 0x7f665803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.618+0000 7f665effd700 1 -- 172.21.15.17:0/1902968643 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f666804d580 con 0x7f666c107af0 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.618+0000 7f66712c9700 1 -- 172.21.15.17:0/1902968643 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f6658038f80 msgr2=0x7f665803b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3007657427 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.618+0000 7f66712c9700 1 --2- 172.21.15.17:0/1902968643 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f6658038f80 0x7f665803b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.623+0000 7f665effd700 1 -- 172.21.15.17:0/1902968643 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6668011e20 con 0x7f666c107af0 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.819+0000 7f66712c9700 1 -- 172.21.15.17:0/1902968643 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f6658038f80 msgr2=0x7f665803b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3007657427 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.819+0000 7f66712c9700 1 --2- 172.21.15.17:0/1902968643 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f6658038f80 0x7f665803b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.893+0000 7f6673d2e700 1 -- 172.21.15.17:0/1902968643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f666c02db30 con 0x7f666c107af0 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.894+0000 7f665effd700 1 -- 172.21.15.17:0/1902968643 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f666800bc30 con 0x7f666c107af0 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.896+0000 7f6673d2e700 1 -- 172.21.15.17:0/1902968643 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f6658038f80 msgr2=0x7f665803b440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.896+0000 7f6673d2e700 1 --2- 172.21.15.17:0/1902968643 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f6658038f80 0x7f665803b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.896+0000 7f6673d2e700 1 -- 172.21.15.17:0/1902968643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 msgr2=0x7f666c19bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.896+0000 7f6673d2e700 1 --2- 172.21.15.17:0/1902968643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 0x7f666c19bdf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66680116a0 tx=0x7f6668011ba0 comp rx=0 tx=0).stop 2024-03-24T02:51:25.510 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.896+0000 7f6673d2e700 1 -- 172.21.15.17:0/1902968643 shutdown_connections 2024-03-24T02:51:25.511 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.896+0000 7f6673d2e700 1 --2- 172.21.15.17:0/1902968643 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f6658038f80 0x7f665803b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:25.511 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.896+0000 7f6673d2e700 1 --2- 172.21.15.17:0/1902968643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f666c107af0 0x7f666c19bdf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:25.511 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.896+0000 7f6673d2e700 1 -- 172.21.15.17:0/1902968643 >> 172.21.15.17:0/1902968643 conn(0x7f666c1030b0 msgr2=0x7f666c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:25.511 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.896+0000 7f6673d2e700 1 -- 172.21.15.17:0/1902968643 shutdown_connections 2024-03-24T02:51:25.511 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:24.896+0000 7f6673d2e700 1 -- 172.21.15.17:0/1902968643 wait complete. 2024-03-24T02:51:25.511 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for the mgr to restart... 2024-03-24T02:51:25.511 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mgr epoch 9... 2024-03-24T02:51:26.455 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:26 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:26.336+0000 7f5f594fd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T02:51:26.808 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:26 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:26.470+0000 7f5f594fd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T02:51:26.808 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:26 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:26.808+0000 7f5f594fd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T02:51:27.102 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:26 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:26.901+0000 7f5f594fd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T02:51:27.102 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:27 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:27.102+0000 7f5f594fd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T02:51:27.371 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:27 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:27.300+0000 7f5f594fd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T02:51:27.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:27 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:27.603+0000 7f5f594fd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T02:51:28.206 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:27 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:27.868+0000 7f5f594fd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T02:51:28.206 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:28 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:28.114+0000 7f5f594fd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T02:51:29.206 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:28 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:28.947+0000 7f5f594fd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T02:51:29.850 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:29.527+0000 7f5f594fd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T02:51:29.850 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:29.627+0000 7f5f594fd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T02:51:29.850 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:51:29.850+0000 7f5f594fd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T02:51:30.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-mon[31926]: Active manager daemon x restarted 2024-03-24T02:51:30.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-mon[31926]: Activating manager daemon x 2024-03-24T02:51:30.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-mon[31926]: osdmap e3: 0 total, 0 up, 0 in 2024-03-24T02:51:30.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-mon[31926]: mgrmap e10: x(active, starting, since 0.00305883s) 2024-03-24T02:51:30.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:51:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T02:51:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T02:51:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T02:51:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T02:51:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-mon[31926]: Manager daemon x is now available 2024-03-24T02:51:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:51:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:51:31.123 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-24T02:51:31.123 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-24T02:51:31.123 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:30 smithi017 ceph-mon[31926]: [24/Mar/2024:02:51:30] ENGINE Bus STARTING 2024-03-24T02:51:31.123 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:30 smithi017 ceph-mon[31926]: [24/Mar/2024:02:51:30] ENGINE Serving on http://172.21.15.17:8765 2024-03-24T02:51:31.123 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:30 smithi017 ceph-mon[31926]: [24/Mar/2024:02:51:30] ENGINE Serving on https://172.21.15.17:7150 2024-03-24T02:51:31.124 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:30 smithi017 ceph-mon[31926]: [24/Mar/2024:02:51:30] ENGINE Bus STARTED 2024-03-24T02:51:31.124 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:30 smithi017 ceph-mon[31926]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-24T02:51:31.124 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:30 smithi017 ceph-mon[31926]: mgrmap e11: x(active, since 1.00584s) 2024-03-24T02:51:31.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-03-24T02:51:31.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-24T02:51:31.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-24T02:51:31.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-03-24T02:51:31.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.966+0000 7f9eb82fe700 1 Processor -- start 2024-03-24T02:51:31.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.966+0000 7f9eb82fe700 1 -- start start 2024-03-24T02:51:31.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.966+0000 7f9eb82fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 0x7f9eb01083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:31.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.966+0000 7f9eb82fe700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0108970 con 0x7f9eb0107f80 2024-03-24T02:51:31.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.966+0000 7f9eb609a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 0x7f9eb01083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:31.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.966+0000 7f9eb609a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 0x7f9eb01083a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45196/0 (socket says 172.21.15.17:45196) 2024-03-24T02:51:31.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.966+0000 7f9eb609a700 1 -- 172.21.15.17:0/884614512 learned_addr learned my addr 172.21.15.17:0/884614512 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.966+0000 7f9eb609a700 1 -- 172.21.15.17:0/884614512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb0108af0 con 0x7f9eb0107f80 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.967+0000 7f9eb609a700 1 --2- 172.21.15.17:0/884614512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 0x7f9eb01083a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40098a0 tx=0x7f9ea4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2609ae0f72dee94c server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.967+0000 7f9eb5098700 1 -- 172.21.15.17:0/884614512 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea4004160 con 0x7f9eb0107f80 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.967+0000 7f9eb5098700 1 -- 172.21.15.17:0/884614512 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ea4004300 con 0x7f9eb0107f80 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.967+0000 7f9eb5098700 1 -- 172.21.15.17:0/884614512 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea4011b20 con 0x7f9eb0107f80 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.967+0000 7f9eb82fe700 1 -- 172.21.15.17:0/884614512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 msgr2=0x7f9eb01083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.967+0000 7f9eb82fe700 1 --2- 172.21.15.17:0/884614512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 0x7f9eb01083a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40098a0 tx=0x7f9ea4009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.967+0000 7f9eb82fe700 1 -- 172.21.15.17:0/884614512 shutdown_connections 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.967+0000 7f9eb82fe700 1 --2- 172.21.15.17:0/884614512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 0x7f9eb01083a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.967+0000 7f9eb82fe700 1 -- 172.21.15.17:0/884614512 >> 172.21.15.17:0/884614512 conn(0x7f9eb01017d0 msgr2=0x7f9eb0103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.968+0000 7f9eb82fe700 1 -- 172.21.15.17:0/884614512 shutdown_connections 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.968+0000 7f9eb82fe700 1 -- 172.21.15.17:0/884614512 wait complete. 2024-03-24T02:51:31.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.968+0000 7f9eb82fe700 1 Processor -- start 2024-03-24T02:51:31.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.968+0000 7f9eb82fe700 1 -- start start 2024-03-24T02:51:31.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.968+0000 7f9eb82fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 0x7f9eb0191bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:31.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.968+0000 7f9eb82fe700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb010af00 con 0x7f9eb0107f80 2024-03-24T02:51:31.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.968+0000 7f9eb609a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 0x7f9eb0191bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:31.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.968+0000 7f9eb609a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 0x7f9eb0191bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45206/0 (socket says 172.21.15.17:45206) 2024-03-24T02:51:31.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.969+0000 7f9eb609a700 1 -- 172.21.15.17:0/3017819791 learned_addr learned my addr 172.21.15.17:0/3017819791 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:31.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.969+0000 7f9eb609a700 1 -- 172.21.15.17:0/3017819791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb0192100 con 0x7f9eb0107f80 2024-03-24T02:51:31.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.969+0000 7f9eb609a700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 0x7f9eb0191bc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40116a0 tx=0x7f9ea4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:31.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.969+0000 7f9ea37fe700 1 -- 172.21.15.17:0/3017819791 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea4003a40 con 0x7f9eb0107f80 2024-03-24T02:51:31.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.969+0000 7f9eb82fe700 1 -- 172.21.15.17:0/3017819791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb01902f0 con 0x7f9eb0107f80 2024-03-24T02:51:31.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.969+0000 7f9eb82fe700 1 -- 172.21.15.17:0/3017819791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb01907b0 con 0x7f9eb0107f80 2024-03-24T02:51:31.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.969+0000 7f9ea37fe700 1 -- 172.21.15.17:0/3017819791 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ea4003be0 con 0x7f9eb0107f80 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.969+0000 7f9ea37fe700 1 -- 172.21.15.17:0/3017819791 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea4004300 con 0x7f9eb0107f80 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.970+0000 7f9ea37fe700 1 -- 172.21.15.17:0/3017819791 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f9ea401a5f0 con 0x7f9eb0107f80 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.970+0000 7f9ea37fe700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 0x7f9e9c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.970+0000 7f9ea37fe700 1 -- 172.21.15.17:0/3017819791 --> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ea4004300 con 0x7f9e9c0390c0 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.970+0000 7f9ea37fe700 1 -- 172.21.15.17:0/3017819791 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9ea404d420 con 0x7f9eb0107f80 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.970+0000 7f9eb5899700 1 -- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 msgr2=0x7f9e9c03b580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3007657427 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:25.970+0000 7f9eb5899700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 0x7f9e9c03b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:26.171+0000 7f9eb5899700 1 -- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 msgr2=0x7f9e9c03b580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3007657427 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:26.171+0000 7f9eb5899700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 0x7f9e9c03b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:26.572+0000 7f9eb5899700 1 -- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 msgr2=0x7f9e9c03b580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3007657427 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:26.572+0000 7f9eb5899700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 0x7f9e9c03b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:27.373+0000 7f9eb5899700 1 -- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 msgr2=0x7f9e9c03b580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3007657427 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:27.373+0000 7f9eb5899700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 0x7f9e9c03b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-24T02:51:31.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:28.974+0000 7f9eb5899700 1 -- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 msgr2=0x7f9e9c03b580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3007657427 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:28.974+0000 7f9eb5899700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 0x7f9e9c03b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:29.866+0000 7f9ea37fe700 1 -- 172.21.15.17:0/3017819791 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mgrmap(e 10) v1 ==== 45886+0+0 (secure 0 0 0) 0x7f9ea401ab60 con 0x7f9eb0107f80 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:29.866+0000 7f9ea37fe700 1 -- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 msgr2=0x7f9e9c03b580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:29.866+0000 7f9ea37fe700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 0x7f9e9c03b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.867+0000 7f9ea37fe700 1 -- 172.21.15.17:0/3017819791 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 11) v1 ==== 46013+0+0 (secure 0 0 0) 0x7f9ea404ebf0 con 0x7f9eb0107f80 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.868+0000 7f9ea37fe700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9e9c03c990 0x7f9e9c03ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.868+0000 7f9ea37fe700 1 -- 172.21.15.17:0/3017819791 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ea4028d00 con 0x7f9e9c03c990 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.868+0000 7f9eb5899700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9e9c03c990 0x7f9e9c03ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.869+0000 7f9eb5899700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9e9c03c990 0x7f9e9c03ed80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9eac003a10 tx=0x7f9eac009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.869+0000 7f9ea37fe700 1 -- 172.21.15.17:0/3017819791 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f9ea4028d00 con 0x7f9e9c03c990 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.876+0000 7f9eb82fe700 1 -- 172.21.15.17:0/3017819791 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f9e94002630 con 0x7f9e9c03c990 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.876+0000 7f9ea37fe700 1 -- 172.21.15.17:0/3017819791 <== mgr.14152 v2:172.21.15.17:6800/2663792883 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f9e94002630 con 0x7f9e9c03c990 2024-03-24T02:51:31.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.876+0000 7f9eb82fe700 1 -- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9e9c03c990 msgr2=0x7f9e9c03ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:31.697 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.876+0000 7f9eb82fe700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9e9c03c990 0x7f9e9c03ed80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9eac003a10 tx=0x7f9eac009270 comp rx=0 tx=0).stop 2024-03-24T02:51:31.697 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.876+0000 7f9eb82fe700 1 -- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 msgr2=0x7f9eb0191bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:31.697 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.876+0000 7f9eb82fe700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 0x7f9eb0191bc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40116a0 tx=0x7f9ea4011ba0 comp rx=0 tx=0).stop 2024-03-24T02:51:31.697 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.877+0000 7f9eb82fe700 1 -- 172.21.15.17:0/3017819791 shutdown_connections 2024-03-24T02:51:31.697 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.877+0000 7f9eb82fe700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9e9c03c990 0x7f9e9c03ed80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:31.697 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.877+0000 7f9eb82fe700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9eb0107f80 0x7f9eb0191bc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:31.697 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.877+0000 7f9eb82fe700 1 --2- 172.21.15.17:0/3017819791 >> [v2:172.21.15.17:6800/3007657427,v1:172.21.15.17:6801/3007657427] conn(0x7f9e9c0390c0 0x7f9e9c03b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:31.697 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.877+0000 7f9eb82fe700 1 -- 172.21.15.17:0/3017819791 >> 172.21.15.17:0/3017819791 conn(0x7f9eb01017d0 msgr2=0x7f9eb0102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:31.697 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.877+0000 7f9eb82fe700 1 -- 172.21.15.17:0/3017819791 shutdown_connections 2024-03-24T02:51:31.697 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:30.877+0000 7f9eb82fe700 1 -- 172.21.15.17:0/3017819791 wait complete. 2024-03-24T02:51:31.697 INFO:teuthology.orchestra.run.smithi017.stdout:mgr epoch 9 is available 2024-03-24T02:51:31.697 INFO:teuthology.orchestra.run.smithi017.stdout:Generating a dashboard self-signed certificate... 2024-03-24T02:51:32.998 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-24T02:51:32.998 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.146+0000 7f8a79d9f700 1 Processor -- start 2024-03-24T02:51:32.998 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.146+0000 7f8a79d9f700 1 -- start start 2024-03-24T02:51:32.998 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.146+0000 7f8a79d9f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 0x7f8a74106c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:32.998 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.146+0000 7f8a79d9f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a74107210 con 0x7f8a74106820 2024-03-24T02:51:32.998 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.146+0000 7f8a78d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 0x7f8a74106c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.146+0000 7f8a78d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 0x7f8a74106c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49570/0 (socket says 172.21.15.17:49570) 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.146+0000 7f8a78d9d700 1 -- 172.21.15.17:0/2324577333 learned_addr learned my addr 172.21.15.17:0/2324577333 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.147+0000 7f8a78d9d700 1 -- 172.21.15.17:0/2324577333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a741079d0 con 0x7f8a74106820 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.147+0000 7f8a78d9d700 1 --2- 172.21.15.17:0/2324577333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 0x7f8a74106c40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8a680098a0 tx=0x7f8a68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61543c063d1217d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.147+0000 7f8a737fe700 1 -- 172.21.15.17:0/2324577333 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a68004160 con 0x7f8a74106820 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.147+0000 7f8a737fe700 1 -- 172.21.15.17:0/2324577333 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a68004300 con 0x7f8a74106820 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.147+0000 7f8a737fe700 1 -- 172.21.15.17:0/2324577333 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a68011b20 con 0x7f8a74106820 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.148+0000 7f8a79d9f700 1 -- 172.21.15.17:0/2324577333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 msgr2=0x7f8a74106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.148+0000 7f8a79d9f700 1 --2- 172.21.15.17:0/2324577333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 0x7f8a74106c40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8a680098a0 tx=0x7f8a68009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.148+0000 7f8a79d9f700 1 -- 172.21.15.17:0/2324577333 shutdown_connections 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.148+0000 7f8a79d9f700 1 --2- 172.21.15.17:0/2324577333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 0x7f8a74106c40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.148+0000 7f8a79d9f700 1 -- 172.21.15.17:0/2324577333 >> 172.21.15.17:0/2324577333 conn(0x7f8a74101de0 msgr2=0x7f8a74104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.148+0000 7f8a79d9f700 1 -- 172.21.15.17:0/2324577333 shutdown_connections 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.148+0000 7f8a79d9f700 1 -- 172.21.15.17:0/2324577333 wait complete. 2024-03-24T02:51:32.999 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.149+0000 7f8a79d9f700 1 Processor -- start 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.149+0000 7f8a79d9f700 1 -- start start 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.149+0000 7f8a79d9f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 0x7f8a7419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.149+0000 7f8a79d9f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7410ad80 con 0x7f8a74106820 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.149+0000 7f8a78d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 0x7f8a7419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.149+0000 7f8a78d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 0x7f8a7419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49582/0 (socket says 172.21.15.17:49582) 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.149+0000 7f8a78d9d700 1 -- 172.21.15.17:0/718668754 learned_addr learned my addr 172.21.15.17:0/718668754 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.150+0000 7f8a78d9d700 1 -- 172.21.15.17:0/718668754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7419af80 con 0x7f8a74106820 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.150+0000 7f8a78d9d700 1 --2- 172.21.15.17:0/718668754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 0x7f8a7419aa40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a680117e0 tx=0x7f8a68011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.150+0000 7f8a71ffb700 1 -- 172.21.15.17:0/718668754 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a68003bf0 con 0x7f8a74106820 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.150+0000 7f8a71ffb700 1 -- 172.21.15.17:0/718668754 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a6801a910 con 0x7f8a74106820 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.150+0000 7f8a79d9f700 1 -- 172.21.15.17:0/718668754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7419b210 con 0x7f8a74106820 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.150+0000 7f8a71ffb700 1 -- 172.21.15.17:0/718668754 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a680249a0 con 0x7f8a74106820 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.150+0000 7f8a79d9f700 1 -- 172.21.15.17:0/718668754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7419b670 con 0x7f8a74106820 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.151+0000 7f8a79d9f700 1 -- 172.21.15.17:0/718668754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a740637c0 con 0x7f8a74106820 2024-03-24T02:51:33.000 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.151+0000 7f8a71ffb700 1 -- 172.21.15.17:0/718668754 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 11) v1 ==== 46013+0+0 (secure 0 0 0) 0x7f8a68022020 con 0x7f8a74106820 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.151+0000 7f8a71ffb700 1 --2- 172.21.15.17:0/718668754 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8a64038e60 0x7f8a6403b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.151+0000 7f8a71ffb700 1 -- 172.21.15.17:0/718668754 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8a6804d090 con 0x7f8a74106820 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.152+0000 7f8a73fff700 1 --2- 172.21.15.17:0/718668754 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8a64038e60 0x7f8a6403b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.152+0000 7f8a73fff700 1 --2- 172.21.15.17:0/718668754 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8a64038e60 0x7f8a6403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8a60006fd0 tx=0x7f8a60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.156+0000 7f8a71ffb700 1 -- 172.21.15.17:0/718668754 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8a68011e20 con 0x7f8a74106820 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.372+0000 7f8a79d9f700 1 -- 172.21.15.17:0/718668754 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a7402db40 con 0x7f8a64038e60 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.415+0000 7f8a71ffb700 1 -- 172.21.15.17:0/718668754 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f8a7402db40 con 0x7f8a64038e60 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.417+0000 7f8a79d9f700 1 -- 172.21.15.17:0/718668754 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8a64038e60 msgr2=0x7f8a6403b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.417+0000 7f8a79d9f700 1 --2- 172.21.15.17:0/718668754 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8a64038e60 0x7f8a6403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8a60006fd0 tx=0x7f8a60006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.417+0000 7f8a79d9f700 1 -- 172.21.15.17:0/718668754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 msgr2=0x7f8a7419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.417+0000 7f8a79d9f700 1 --2- 172.21.15.17:0/718668754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 0x7f8a7419aa40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a680117e0 tx=0x7f8a68011c10 comp rx=0 tx=0).stop 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.417+0000 7f8a79d9f700 1 -- 172.21.15.17:0/718668754 shutdown_connections 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.417+0000 7f8a79d9f700 1 --2- 172.21.15.17:0/718668754 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8a64038e60 0x7f8a6403b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:33.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.417+0000 7f8a79d9f700 1 --2- 172.21.15.17:0/718668754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a74106820 0x7f8a7419aa40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:33.002 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.417+0000 7f8a79d9f700 1 -- 172.21.15.17:0/718668754 >> 172.21.15.17:0/718668754 conn(0x7f8a74101de0 msgr2=0x7f8a74102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:33.002 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.417+0000 7f8a79d9f700 1 -- 172.21.15.17:0/718668754 shutdown_connections 2024-03-24T02:51:33.002 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:32.417+0000 7f8a79d9f700 1 -- 172.21.15.17:0/718668754 wait complete. 2024-03-24T02:51:33.002 INFO:teuthology.orchestra.run.smithi017.stdout:Creating initial admin user... 2024-03-24T02:51:33.116 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:32 smithi017 ceph-mon[31926]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-24T02:51:33.117 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:33.117 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:33.117 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:33.985 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:33 smithi017 ceph-mon[31926]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:51:33.985 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:33 smithi017 ceph-mon[31926]: mgrmap e12: x(active, since 2s) 2024-03-24T02:51:33.985 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:34.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$EJgcMi9nky1w9gpP1l.XMu/Wjwzu8lVt08zhilNJdAx/Igf/0Q/Yq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711248693, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-24T02:51:34.692 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.468+0000 7f1f665c0700 1 Processor -- start 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.468+0000 7f1f665c0700 1 -- start start 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.468+0000 7f1f665c0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 0x7f1f60106e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.468+0000 7f1f665c0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60107400 con 0x7f1f60106a10 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.468+0000 7f1f5ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 0x7f1f60106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.468+0000 7f1f5ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 0x7f1f60106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49588/0 (socket says 172.21.15.17:49588) 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.468+0000 7f1f5ffff700 1 -- 172.21.15.17:0/3119261045 learned_addr learned my addr 172.21.15.17:0/3119261045 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.468+0000 7f1f5ffff700 1 -- 172.21.15.17:0/3119261045 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f60107c10 con 0x7f1f60106a10 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.469+0000 7f1f5ffff700 1 --2- 172.21.15.17:0/3119261045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 0x7f1f60106e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1f500098a0 tx=0x7f1f50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be4097ac0c71a01 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.469+0000 7f1f5effd700 1 -- 172.21.15.17:0/3119261045 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f50004160 con 0x7f1f60106a10 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.469+0000 7f1f5effd700 1 -- 172.21.15.17:0/3119261045 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f50004300 con 0x7f1f60106a10 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.469+0000 7f1f5effd700 1 -- 172.21.15.17:0/3119261045 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f50011b20 con 0x7f1f60106a10 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.469+0000 7f1f665c0700 1 -- 172.21.15.17:0/3119261045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 msgr2=0x7f1f60106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:34.693 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.469+0000 7f1f665c0700 1 --2- 172.21.15.17:0/3119261045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 0x7f1f60106e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1f500098a0 tx=0x7f1f50009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.469+0000 7f1f665c0700 1 -- 172.21.15.17:0/3119261045 shutdown_connections 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.469+0000 7f1f665c0700 1 --2- 172.21.15.17:0/3119261045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 0x7f1f60106e30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.469+0000 7f1f665c0700 1 -- 172.21.15.17:0/3119261045 >> 172.21.15.17:0/3119261045 conn(0x7f1f60101fb0 msgr2=0x7f1f601043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.469+0000 7f1f665c0700 1 -- 172.21.15.17:0/3119261045 shutdown_connections 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.469+0000 7f1f665c0700 1 -- 172.21.15.17:0/3119261045 wait complete. 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.470+0000 7f1f665c0700 1 Processor -- start 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.470+0000 7f1f665c0700 1 -- start start 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.470+0000 7f1f665c0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 0x7f1f60191bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.470+0000 7f1f665c0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6010afc0 con 0x7f1f60106a10 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.470+0000 7f1f5ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 0x7f1f60191bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.470+0000 7f1f5ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 0x7f1f60191bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49604/0 (socket says 172.21.15.17:49604) 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.470+0000 7f1f5ffff700 1 -- 172.21.15.17:0/3020286281 learned_addr learned my addr 172.21.15.17:0/3020286281 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:34.694 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.471+0000 7f1f5ffff700 1 -- 172.21.15.17:0/3020286281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f60192100 con 0x7f1f60106a10 2024-03-24T02:51:34.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.471+0000 7f1f5ffff700 1 --2- 172.21.15.17:0/3020286281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 0x7f1f60191bc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f50000c00 tx=0x7f1f50011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:34.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.471+0000 7f1f5d7fa700 1 -- 172.21.15.17:0/3020286281 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f50003b60 con 0x7f1f60106a10 2024-03-24T02:51:34.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.471+0000 7f1f665c0700 1 -- 172.21.15.17:0/3020286281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f601902f0 con 0x7f1f60106a10 2024-03-24T02:51:34.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.471+0000 7f1f5d7fa700 1 -- 172.21.15.17:0/3020286281 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f5001a910 con 0x7f1f60106a10 2024-03-24T02:51:34.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.471+0000 7f1f5d7fa700 1 -- 172.21.15.17:0/3020286281 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f50024930 con 0x7f1f60106a10 2024-03-24T02:51:34.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.471+0000 7f1f665c0700 1 -- 172.21.15.17:0/3020286281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f601907b0 con 0x7f1f60106a10 2024-03-24T02:51:34.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.472+0000 7f1f5d7fa700 1 -- 172.21.15.17:0/3020286281 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f1f50022020 con 0x7f1f60106a10 2024-03-24T02:51:34.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.472+0000 7f1f5d7fa700 1 --2- 172.21.15.17:0/3020286281 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1f48038f70 0x7f1f4803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:34.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.472+0000 7f1f5d7fa700 1 -- 172.21.15.17:0/3020286281 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1f5004d1c0 con 0x7f1f60106a10 2024-03-24T02:51:34.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.472+0000 7f1f665c0700 1 -- 172.21.15.17:0/3020286281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f600637c0 con 0x7f1f60106a10 2024-03-24T02:51:34.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.473+0000 7f1f5f7fe700 1 --2- 172.21.15.17:0/3020286281 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1f48038f70 0x7f1f4803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:34.695 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.473+0000 7f1f5f7fe700 1 --2- 172.21.15.17:0/3020286281 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1f48038f70 0x7f1f4803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f54006fd0 tx=0x7f1f54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.478+0000 7f1f5d7fa700 1 -- 172.21.15.17:0/3020286281 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f500172b0 con 0x7f1f60106a10 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.740+0000 7f1f665c0700 1 -- 172.21.15.17:0/3020286281 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1f60195560 con 0x7f1f48038f70 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.986+0000 7f1f5d7fa700 1 -- 172.21.15.17:0/3020286281 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f1f60195560 con 0x7f1f48038f70 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.988+0000 7f1f665c0700 1 -- 172.21.15.17:0/3020286281 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1f48038f70 msgr2=0x7f1f4803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.988+0000 7f1f665c0700 1 --2- 172.21.15.17:0/3020286281 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1f48038f70 0x7f1f4803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f54006fd0 tx=0x7f1f54006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.988+0000 7f1f665c0700 1 -- 172.21.15.17:0/3020286281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 msgr2=0x7f1f60191bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.988+0000 7f1f665c0700 1 --2- 172.21.15.17:0/3020286281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 0x7f1f60191bc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f50000c00 tx=0x7f1f50011c10 comp rx=0 tx=0).stop 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.988+0000 7f1f665c0700 1 -- 172.21.15.17:0/3020286281 shutdown_connections 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.988+0000 7f1f665c0700 1 --2- 172.21.15.17:0/3020286281 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1f48038f70 0x7f1f4803b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.988+0000 7f1f665c0700 1 --2- 172.21.15.17:0/3020286281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f60106a10 0x7f1f60191bc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.988+0000 7f1f665c0700 1 -- 172.21.15.17:0/3020286281 >> 172.21.15.17:0/3020286281 conn(0x7f1f60101fb0 msgr2=0x7f1f60102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.988+0000 7f1f665c0700 1 -- 172.21.15.17:0/3020286281 shutdown_connections 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:33.988+0000 7f1f665c0700 1 -- 172.21.15.17:0/3020286281 wait complete. 2024-03-24T02:51:34.696 INFO:teuthology.orchestra.run.smithi017.stdout:Fetching dashboard port number... 2024-03-24T02:51:35.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:34 smithi017 ceph-mon[31926]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:51:35.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:34 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:36.076 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:35 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2828579265' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-24T02:51:36.083 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 8443 2024-03-24T02:51:36.083 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.113+0000 7f95945eb700 1 Processor -- start 2024-03-24T02:51:36.083 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.113+0000 7f95945eb700 1 -- start start 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.113+0000 7f95945eb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 0x7f958c10aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.113+0000 7f95945eb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c07bdb0 con 0x7f958c108630 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.113+0000 7f9592387700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 0x7f958c10aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.114+0000 7f9592387700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 0x7f958c10aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49614/0 (socket says 172.21.15.17:49614) 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.114+0000 7f9592387700 1 -- 172.21.15.17:0/1233305901 learned_addr learned my addr 172.21.15.17:0/1233305901 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.114+0000 7f9592387700 1 -- 172.21.15.17:0/1233305901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c10af60 con 0x7f958c108630 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.114+0000 7f9592387700 1 --2- 172.21.15.17:0/1233305901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 0x7f958c10aa20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f95880098a0 tx=0x7f9588009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edadf518b8bf56b server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.114+0000 7f9591385700 1 -- 172.21.15.17:0/1233305901 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9588004160 con 0x7f958c108630 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.114+0000 7f9591385700 1 -- 172.21.15.17:0/1233305901 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9588004300 con 0x7f958c108630 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.114+0000 7f9591385700 1 -- 172.21.15.17:0/1233305901 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9588011b20 con 0x7f958c108630 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.115+0000 7f95945eb700 1 -- 172.21.15.17:0/1233305901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 msgr2=0x7f958c10aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.115+0000 7f95945eb700 1 --2- 172.21.15.17:0/1233305901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 0x7f958c10aa20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f95880098a0 tx=0x7f9588009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.115+0000 7f95945eb700 1 -- 172.21.15.17:0/1233305901 shutdown_connections 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.115+0000 7f95945eb700 1 --2- 172.21.15.17:0/1233305901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 0x7f958c10aa20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:36.084 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.115+0000 7f95945eb700 1 -- 172.21.15.17:0/1233305901 >> 172.21.15.17:0/1233305901 conn(0x7f958c102000 msgr2=0x7f958c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.115+0000 7f95945eb700 1 -- 172.21.15.17:0/1233305901 shutdown_connections 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.115+0000 7f95945eb700 1 -- 172.21.15.17:0/1233305901 wait complete. 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.115+0000 7f95945eb700 1 Processor -- start 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.116+0000 7f95945eb700 1 -- start start 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.116+0000 7f95945eb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 0x7f958c107eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.116+0000 7f95945eb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c10bf00 con 0x7f958c108630 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.116+0000 7f9592387700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 0x7f958c107eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.116+0000 7f9592387700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 0x7f958c107eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49624/0 (socket says 172.21.15.17:49624) 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.116+0000 7f9592387700 1 -- 172.21.15.17:0/2828579265 learned_addr learned my addr 172.21.15.17:0/2828579265 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.116+0000 7f9592387700 1 -- 172.21.15.17:0/2828579265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c1083f0 con 0x7f958c108630 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.116+0000 7f9592387700 1 --2- 172.21.15.17:0/2828579265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 0x7f958c107eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9588011b60 tx=0x7f9588011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.116+0000 7f95837fe700 1 -- 172.21.15.17:0/2828579265 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9588003b60 con 0x7f958c108630 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.116+0000 7f95837fe700 1 -- 172.21.15.17:0/2828579265 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f958801a910 con 0x7f958c108630 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.116+0000 7f95945eb700 1 -- 172.21.15.17:0/2828579265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958c1065e0 con 0x7f958c108630 2024-03-24T02:51:36.085 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.116+0000 7f95837fe700 1 -- 172.21.15.17:0/2828579265 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9588024930 con 0x7f958c108630 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.117+0000 7f95945eb700 1 -- 172.21.15.17:0/2828579265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958c106aa0 con 0x7f958c108630 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.117+0000 7f95945eb700 1 -- 172.21.15.17:0/2828579265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958c0637c0 con 0x7f958c108630 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.118+0000 7f95837fe700 1 -- 172.21.15.17:0/2828579265 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f9588022020 con 0x7f958c108630 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.118+0000 7f95837fe700 1 --2- 172.21.15.17:0/2828579265 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9578038fd0 0x7f957803b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.118+0000 7f95837fe700 1 -- 172.21.15.17:0/2828579265 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f958804d2a0 con 0x7f958c108630 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.118+0000 7f9591b86700 1 --2- 172.21.15.17:0/2828579265 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9578038fd0 0x7f957803b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.118+0000 7f9591b86700 1 --2- 172.21.15.17:0/2828579265 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9578038fd0 0x7f957803b490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f957c006fd0 tx=0x7f957c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.123+0000 7f95837fe700 1 -- 172.21.15.17:0/2828579265 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f958802d770 con 0x7f958c108630 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.328+0000 7f95945eb700 1 -- 172.21.15.17:0/2828579265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f958c106d20 con 0x7f958c108630 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.328+0000 7f95837fe700 1 -- 172.21.15.17:0/2828579265 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f9588024ad0 con 0x7f958c108630 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.330+0000 7f95945eb700 1 -- 172.21.15.17:0/2828579265 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9578038fd0 msgr2=0x7f957803b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.330+0000 7f95945eb700 1 --2- 172.21.15.17:0/2828579265 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9578038fd0 0x7f957803b490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f957c006fd0 tx=0x7f957c006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.330+0000 7f95945eb700 1 -- 172.21.15.17:0/2828579265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 msgr2=0x7f958c107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:36.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.330+0000 7f95945eb700 1 --2- 172.21.15.17:0/2828579265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 0x7f958c107eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9588011b60 tx=0x7f9588011c40 comp rx=0 tx=0).stop 2024-03-24T02:51:36.087 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.330+0000 7f95945eb700 1 -- 172.21.15.17:0/2828579265 shutdown_connections 2024-03-24T02:51:36.087 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.330+0000 7f95945eb700 1 --2- 172.21.15.17:0/2828579265 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9578038fd0 0x7f957803b490 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:36.087 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.330+0000 7f95945eb700 1 --2- 172.21.15.17:0/2828579265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c108630 0x7f958c107eb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:36.087 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.330+0000 7f95945eb700 1 -- 172.21.15.17:0/2828579265 >> 172.21.15.17:0/2828579265 conn(0x7f958c102000 msgr2=0x7f958c102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:36.087 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.331+0000 7f95945eb700 1 -- 172.21.15.17:0/2828579265 shutdown_connections 2024-03-24T02:51:36.087 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:35.331+0000 7f95945eb700 1 -- 172.21.15.17:0/2828579265 wait complete. 2024-03-24T02:51:36.090 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T02:51:36.090 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout disabled 2024-03-24T02:51:36.097 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T02:51:36.097 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2024-03-24T02:51:36.097 INFO:teuthology.orchestra.run.smithi017.stdout:firewalld.service is not enabled 2024-03-24T02:51:36.097 INFO:teuthology.orchestra.run.smithi017.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-24T02:51:36.099 INFO:teuthology.orchestra.run.smithi017.stdout:Ceph Dashboard is now available at: 2024-03-24T02:51:36.099 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:51:36.099 INFO:teuthology.orchestra.run.smithi017.stdout: URL: https://smithi017.front.sepia.ceph.com:8443/ 2024-03-24T02:51:36.099 INFO:teuthology.orchestra.run.smithi017.stdout: User: admin 2024-03-24T02:51:36.099 INFO:teuthology.orchestra.run.smithi017.stdout: Password: suic2fzdk4 2024-03-24T02:51:36.099 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:51:36.099 INFO:teuthology.orchestra.run.smithi017.stdout:Saving cluster configuration to /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config directory 2024-03-24T02:51:36.100 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling autotune for osd_memory_target 2024-03-24T02:51:37.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.543+0000 7fe15273f700 1 Processor -- start 2024-03-24T02:51:37.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.543+0000 7fe15273f700 1 -- start start 2024-03-24T02:51:37.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.543+0000 7fe15273f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 0x7fe14c10a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:37.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.543+0000 7fe15273f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c07be90 con 0x7fe14c108550 2024-03-24T02:51:37.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.543+0000 7fe14bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 0x7fe14c10a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:37.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.543+0000 7fe14bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 0x7fe14c10a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49630/0 (socket says 172.21.15.17:49630) 2024-03-24T02:51:37.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.543+0000 7fe14bfff700 1 -- 172.21.15.17:0/1202380256 learned_addr learned my addr 172.21.15.17:0/1202380256 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:37.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.543+0000 7fe14bfff700 1 -- 172.21.15.17:0/1202380256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c10ae80 con 0x7fe14c108550 2024-03-24T02:51:37.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.543+0000 7fe14bfff700 1 --2- 172.21.15.17:0/1202380256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 0x7fe14c10a940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0098a0 tx=0x7fe13c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8cc45ffd2e96cc server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.543+0000 7fe14affd700 1 -- 172.21.15.17:0/1202380256 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe13c004160 con 0x7fe14c108550 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.544+0000 7fe14affd700 1 -- 172.21.15.17:0/1202380256 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe13c004300 con 0x7fe14c108550 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.544+0000 7fe14affd700 1 -- 172.21.15.17:0/1202380256 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe13c011b20 con 0x7fe14c108550 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.544+0000 7fe15273f700 1 -- 172.21.15.17:0/1202380256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 msgr2=0x7fe14c10a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.544+0000 7fe15273f700 1 --2- 172.21.15.17:0/1202380256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 0x7fe14c10a940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0098a0 tx=0x7fe13c009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.544+0000 7fe15273f700 1 -- 172.21.15.17:0/1202380256 shutdown_connections 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.544+0000 7fe15273f700 1 --2- 172.21.15.17:0/1202380256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 0x7fe14c10a940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.544+0000 7fe15273f700 1 -- 172.21.15.17:0/1202380256 >> 172.21.15.17:0/1202380256 conn(0x7fe14c102000 msgr2=0x7fe14c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.544+0000 7fe15273f700 1 -- 172.21.15.17:0/1202380256 shutdown_connections 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.544+0000 7fe15273f700 1 -- 172.21.15.17:0/1202380256 wait complete. 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.545+0000 7fe15273f700 1 Processor -- start 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.545+0000 7fe15273f700 1 -- start start 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.545+0000 7fe15273f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 0x7fe14c107dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.545+0000 7fe15273f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c10bef0 con 0x7fe14c108550 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.545+0000 7fe14bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 0x7fe14c107dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:37.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.545+0000 7fe14bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 0x7fe14c107dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49646/0 (socket says 172.21.15.17:49646) 2024-03-24T02:51:37.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.545+0000 7fe14bfff700 1 -- 172.21.15.17:0/3971374758 learned_addr learned my addr 172.21.15.17:0/3971374758 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:37.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.545+0000 7fe14bfff700 1 -- 172.21.15.17:0/3971374758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c108310 con 0x7fe14c108550 2024-03-24T02:51:37.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.546+0000 7fe14bfff700 1 --2- 172.21.15.17:0/3971374758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 0x7fe14c107dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe13c011b60 tx=0x7fe13c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:37.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.546+0000 7fe1497fa700 1 -- 172.21.15.17:0/3971374758 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe13c003b60 con 0x7fe14c108550 2024-03-24T02:51:37.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.546+0000 7fe1497fa700 1 -- 172.21.15.17:0/3971374758 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe13c01a910 con 0x7fe14c108550 2024-03-24T02:51:37.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.546+0000 7fe15273f700 1 -- 172.21.15.17:0/3971374758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14c106500 con 0x7fe14c108550 2024-03-24T02:51:37.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.546+0000 7fe1497fa700 1 -- 172.21.15.17:0/3971374758 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe13c024930 con 0x7fe14c108550 2024-03-24T02:51:37.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.546+0000 7fe15273f700 1 -- 172.21.15.17:0/3971374758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14c1069c0 con 0x7fe14c108550 2024-03-24T02:51:37.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.547+0000 7fe1497fa700 1 -- 172.21.15.17:0/3971374758 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fe13c022020 con 0x7fe14c108550 2024-03-24T02:51:37.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.547+0000 7fe15273f700 1 -- 172.21.15.17:0/3971374758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14c0637c0 con 0x7fe14c108550 2024-03-24T02:51:37.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.547+0000 7fe1497fa700 1 --2- 172.21.15.17:0/3971374758 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe134038fd0 0x7fe13403b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:37.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.547+0000 7fe1497fa700 1 -- 172.21.15.17:0/3971374758 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe13c04d2b0 con 0x7fe14c108550 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.547+0000 7fe14b7fe700 1 --2- 172.21.15.17:0/3971374758 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe134038fd0 0x7fe13403b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.547+0000 7fe14b7fe700 1 --2- 172.21.15.17:0/3971374758 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe134038fd0 0x7fe13403b490 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe140006fd0 tx=0x7fe140006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.552+0000 7fe1497fa700 1 -- 172.21.15.17:0/3971374758 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe13c02d770 con 0x7fe14c108550 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.758+0000 7fe15273f700 1 -- 172.21.15.17:0/3971374758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fe14c106c40 con 0x7fe14c108550 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.759+0000 7fe1497fa700 1 -- 172.21.15.17:0/3971374758 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fe13c024ad0 con 0x7fe14c108550 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.761+0000 7fe15273f700 1 -- 172.21.15.17:0/3971374758 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe134038fd0 msgr2=0x7fe13403b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.761+0000 7fe15273f700 1 --2- 172.21.15.17:0/3971374758 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe134038fd0 0x7fe13403b490 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe140006fd0 tx=0x7fe140006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.761+0000 7fe15273f700 1 -- 172.21.15.17:0/3971374758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 msgr2=0x7fe14c107dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.761+0000 7fe15273f700 1 --2- 172.21.15.17:0/3971374758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 0x7fe14c107dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe13c011b60 tx=0x7fe13c011c40 comp rx=0 tx=0).stop 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.761+0000 7fe15273f700 1 -- 172.21.15.17:0/3971374758 shutdown_connections 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.761+0000 7fe15273f700 1 --2- 172.21.15.17:0/3971374758 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe134038fd0 0x7fe13403b490 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.761+0000 7fe15273f700 1 --2- 172.21.15.17:0/3971374758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe14c108550 0x7fe14c107dd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.761+0000 7fe15273f700 1 -- 172.21.15.17:0/3971374758 >> 172.21.15.17:0/3971374758 conn(0x7fe14c102000 msgr2=0x7fe14c102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.761+0000 7fe15273f700 1 -- 172.21.15.17:0/3971374758 shutdown_connections 2024-03-24T02:51:37.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:36.761+0000 7fe15273f700 1 -- 172.21.15.17:0/3971374758 wait complete. 2024-03-24T02:51:37.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:37 smithi017 ceph-mon[31926]: mgrmap e13: x(active, since 6s) 2024-03-24T02:51:38.813 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.882+0000 7f132d7d5700 1 Processor -- start 2024-03-24T02:51:38.813 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.882+0000 7f132d7d5700 1 -- start start 2024-03-24T02:51:38.813 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.883+0000 7f132d7d5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 0x7f1328107f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:38.813 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.883+0000 7f132d7d5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13281084e0 con 0x7f1328107af0 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.883+0000 7f1326ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 0x7f1328107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.883+0000 7f1326ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 0x7f1328107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49654/0 (socket says 172.21.15.17:49654) 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.883+0000 7f1326ffd700 1 -- 172.21.15.17:0/2534300616 learned_addr learned my addr 172.21.15.17:0/2534300616 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.883+0000 7f1326ffd700 1 -- 172.21.15.17:0/2534300616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1328108cf0 con 0x7f1328107af0 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.883+0000 7f1326ffd700 1 --2- 172.21.15.17:0/2534300616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 0x7f1328107f10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13180098a0 tx=0x7f1318009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59c94435f24d19d server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.883+0000 7f1325ffb700 1 -- 172.21.15.17:0/2534300616 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1318004160 con 0x7f1328107af0 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.883+0000 7f1325ffb700 1 -- 172.21.15.17:0/2534300616 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1318004300 con 0x7f1328107af0 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.883+0000 7f1325ffb700 1 -- 172.21.15.17:0/2534300616 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1318011b20 con 0x7f1328107af0 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.884+0000 7f132d7d5700 1 -- 172.21.15.17:0/2534300616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 msgr2=0x7f1328107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.884+0000 7f132d7d5700 1 --2- 172.21.15.17:0/2534300616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 0x7f1328107f10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13180098a0 tx=0x7f1318009bb0 comp rx=0 tx=0).stop 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.884+0000 7f132d7d5700 1 -- 172.21.15.17:0/2534300616 shutdown_connections 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.884+0000 7f132d7d5700 1 --2- 172.21.15.17:0/2534300616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 0x7f1328107f10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.884+0000 7f132d7d5700 1 -- 172.21.15.17:0/2534300616 >> 172.21.15.17:0/2534300616 conn(0x7f13281030b0 msgr2=0x7f13281054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:38.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.884+0000 7f132d7d5700 1 -- 172.21.15.17:0/2534300616 shutdown_connections 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.884+0000 7f132d7d5700 1 -- 172.21.15.17:0/2534300616 wait complete. 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.885+0000 7f132d7d5700 1 Processor -- start 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.885+0000 7f132d7d5700 1 -- start start 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.885+0000 7f132d7d5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 0x7f132819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.885+0000 7f132d7d5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132810c0a0 con 0x7f1328107af0 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.885+0000 7f1326ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 0x7f132819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.885+0000 7f1326ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 0x7f132819bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49668/0 (socket says 172.21.15.17:49668) 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.885+0000 7f1326ffd700 1 -- 172.21.15.17:0/386745991 learned_addr learned my addr 172.21.15.17:0/386745991 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.885+0000 7f1326ffd700 1 -- 172.21.15.17:0/386745991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132819c330 con 0x7f1328107af0 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.886+0000 7f1326ffd700 1 --2- 172.21.15.17:0/386745991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 0x7f132819bdf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f13180116a0 tx=0x7f1318011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.886+0000 7f131ffff700 1 -- 172.21.15.17:0/386745991 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1318003a40 con 0x7f1328107af0 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.886+0000 7f131ffff700 1 -- 172.21.15.17:0/386745991 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1318003be0 con 0x7f1328107af0 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.886+0000 7f132d7d5700 1 -- 172.21.15.17:0/386745991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f132819c5c0 con 0x7f1328107af0 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.886+0000 7f131ffff700 1 -- 172.21.15.17:0/386745991 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f131801a720 con 0x7f1328107af0 2024-03-24T02:51:38.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.886+0000 7f132d7d5700 1 -- 172.21.15.17:0/386745991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132819ca20 con 0x7f1328107af0 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.887+0000 7f131ffff700 1 -- 172.21.15.17:0/386745991 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f131801a9f0 con 0x7f1328107af0 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.887+0000 7f132d7d5700 1 -- 172.21.15.17:0/386745991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13280637c0 con 0x7f1328107af0 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.887+0000 7f131ffff700 1 --2- 172.21.15.17:0/386745991 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1314039020 0x7f131403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.887+0000 7f131ffff700 1 -- 172.21.15.17:0/386745991 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f131804d7d0 con 0x7f1328107af0 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.887+0000 7f13267fc700 1 --2- 172.21.15.17:0/386745991 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1314039020 0x7f131403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.887+0000 7f13267fc700 1 --2- 172.21.15.17:0/386745991 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1314039020 0x7f131403b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1310006fd0 tx=0x7f1310006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:37.892+0000 7f131ffff700 1 -- 172.21.15.17:0/386745991 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1318011e20 con 0x7f1328107af0 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:38.209+0000 7f132d7d5700 1 -- 172.21.15.17:0/386745991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f132802dad0 con 0x7f1328107af0 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:38.212+0000 7f131ffff700 1 -- 172.21.15.17:0/386745991 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f131801ade0 con 0x7f1328107af0 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:38.214+0000 7f132d7d5700 1 -- 172.21.15.17:0/386745991 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1314039020 msgr2=0x7f131403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:38.214+0000 7f132d7d5700 1 --2- 172.21.15.17:0/386745991 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1314039020 0x7f131403b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1310006fd0 tx=0x7f1310006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:38.214+0000 7f132d7d5700 1 -- 172.21.15.17:0/386745991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 msgr2=0x7f132819bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:38.214+0000 7f132d7d5700 1 --2- 172.21.15.17:0/386745991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 0x7f132819bdf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f13180116a0 tx=0x7f1318011ba0 comp rx=0 tx=0).stop 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:38.214+0000 7f132d7d5700 1 -- 172.21.15.17:0/386745991 shutdown_connections 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:38.214+0000 7f132d7d5700 1 --2- 172.21.15.17:0/386745991 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1314039020 0x7f131403b4e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:38.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:38.214+0000 7f132d7d5700 1 --2- 172.21.15.17:0/386745991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1328107af0 0x7f132819bdf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:38.214+0000 7f132d7d5700 1 -- 172.21.15.17:0/386745991 >> 172.21.15.17:0/386745991 conn(0x7f13281030b0 msgr2=0x7f1328103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:38.214+0000 7f132d7d5700 1 -- 172.21.15.17:0/386745991 shutdown_connections 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-03-24T02:51:38.214+0000 7f132d7d5700 1 -- 172.21.15.17:0/386745991 wait complete. 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout:Or, if you are only running a single cluster on this host: 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: ceph telemetry on 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:51:38.817 INFO:teuthology.orchestra.run.smithi017.stdout:For more information see: 2024-03-24T02:51:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:51:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-24T02:51:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:51:38.818 INFO:teuthology.orchestra.run.smithi017.stdout:Bootstrap complete. 2024-03-24T02:51:38.861 INFO:tasks.cephadm:Fetching config... 2024-03-24T02:51:38.862 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:51:38.862 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-24T02:51:38.878 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-24T02:51:38.878 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:51:38.878 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-24T02:51:38.933 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-24T02:51:38.933 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:51:38.933 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-24T02:51:39.000 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-24T02:51:39.000 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:51:39.000 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-24T02:51:39.054 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-24T02:51:39.055 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4lFKfDQCQvBQuy3vj5h7NI+H7m+BxgALhKKgys5uJU/9U5Ih0DqN6SpoTFCu7I/4pOT19nOONPdtc51M0zWRqW9oDWIOJRAw1uSgo0Z1b/Gr5uH1H3vyF8GxBCSNEz6t5FI3KWcj/8Pka7ngQHosu5u3ucWazg/FoaPfCs9tzE1xhi5OBA5g53PIE6wkaUpYIlwqQfQz9LAnayND8J6ho2yeaB3YPtUZX5OQ5Ck+fZAaMH+2QxH7ed42Ih9NYqcO4SBHMgHqcQsqzpX9D+tPdShzRohbTjS2qhg+K4uWAARO+9598ZioUwBxnUWjbPsXcC53x8tm6jEv9RoETIW/AEC/9wON5q0McTbo69fuTpbpPIWAxB6IEgql/lOn1oP70xIJpXtmJAW7csH6VxvRpzbyiY02i9TPRR2jXCwdnnobb4x3IgtXqMpL2Nn2xVlsWteCkpqAk9JUZs0mMP9J4y+Sd7pc+16HPcOnM9vIJcP3aE5eKccfhuXTRv/7S4I8= ceph-2a682800-e989-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-24T02:51:39.137 INFO:teuthology.orchestra.run.smithi017.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4lFKfDQCQvBQuy3vj5h7NI+H7m+BxgALhKKgys5uJU/9U5Ih0DqN6SpoTFCu7I/4pOT19nOONPdtc51M0zWRqW9oDWIOJRAw1uSgo0Z1b/Gr5uH1H3vyF8GxBCSNEz6t5FI3KWcj/8Pka7ngQHosu5u3ucWazg/FoaPfCs9tzE1xhi5OBA5g53PIE6wkaUpYIlwqQfQz9LAnayND8J6ho2yeaB3YPtUZX5OQ5Ck+fZAaMH+2QxH7ed42Ih9NYqcO4SBHMgHqcQsqzpX9D+tPdShzRohbTjS2qhg+K4uWAARO+9598ZioUwBxnUWjbPsXcC53x8tm6jEv9RoETIW/AEC/9wON5q0McTbo69fuTpbpPIWAxB6IEgql/lOn1oP70xIJpXtmJAW7csH6VxvRpzbyiY02i9TPRR2jXCwdnnobb4x3IgtXqMpL2Nn2xVlsWteCkpqAk9JUZs0mMP9J4y+Sd7pc+16HPcOnM9vIJcP3aE5eKccfhuXTRv/7S4I8= ceph-2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:51:39.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4lFKfDQCQvBQuy3vj5h7NI+H7m+BxgALhKKgys5uJU/9U5Ih0DqN6SpoTFCu7I/4pOT19nOONPdtc51M0zWRqW9oDWIOJRAw1uSgo0Z1b/Gr5uH1H3vyF8GxBCSNEz6t5FI3KWcj/8Pka7ngQHosu5u3ucWazg/FoaPfCs9tzE1xhi5OBA5g53PIE6wkaUpYIlwqQfQz9LAnayND8J6ho2yeaB3YPtUZX5OQ5Ck+fZAaMH+2QxH7ed42Ih9NYqcO4SBHMgHqcQsqzpX9D+tPdShzRohbTjS2qhg+K4uWAARO+9598ZioUwBxnUWjbPsXcC53x8tm6jEv9RoETIW/AEC/9wON5q0McTbo69fuTpbpPIWAxB6IEgql/lOn1oP70xIJpXtmJAW7csH6VxvRpzbyiY02i9TPRR2jXCwdnnobb4x3IgtXqMpL2Nn2xVlsWteCkpqAk9JUZs0mMP9J4y+Sd7pc+16HPcOnM9vIJcP3aE5eKccfhuXTRv/7S4I8= ceph-2a682800-e989-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-24T02:51:39.205 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4lFKfDQCQvBQuy3vj5h7NI+H7m+BxgALhKKgys5uJU/9U5Ih0DqN6SpoTFCu7I/4pOT19nOONPdtc51M0zWRqW9oDWIOJRAw1uSgo0Z1b/Gr5uH1H3vyF8GxBCSNEz6t5FI3KWcj/8Pka7ngQHosu5u3ucWazg/FoaPfCs9tzE1xhi5OBA5g53PIE6wkaUpYIlwqQfQz9LAnayND8J6ho2yeaB3YPtUZX5OQ5Ck+fZAaMH+2QxH7ed42Ih9NYqcO4SBHMgHqcQsqzpX9D+tPdShzRohbTjS2qhg+K4uWAARO+9598ZioUwBxnUWjbPsXcC53x8tm6jEv9RoETIW/AEC/9wON5q0McTbo69fuTpbpPIWAxB6IEgql/lOn1oP70xIJpXtmJAW7csH6VxvRpzbyiY02i9TPRR2jXCwdnnobb4x3IgtXqMpL2Nn2xVlsWteCkpqAk9JUZs0mMP9J4y+Sd7pc+16HPcOnM9vIJcP3aE5eKccfhuXTRv/7S4I8= ceph-2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:51:39.224 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4lFKfDQCQvBQuy3vj5h7NI+H7m+BxgALhKKgys5uJU/9U5Ih0DqN6SpoTFCu7I/4pOT19nOONPdtc51M0zWRqW9oDWIOJRAw1uSgo0Z1b/Gr5uH1H3vyF8GxBCSNEz6t5FI3KWcj/8Pka7ngQHosu5u3ucWazg/FoaPfCs9tzE1xhi5OBA5g53PIE6wkaUpYIlwqQfQz9LAnayND8J6ho2yeaB3YPtUZX5OQ5Ck+fZAaMH+2QxH7ed42Ih9NYqcO4SBHMgHqcQsqzpX9D+tPdShzRohbTjS2qhg+K4uWAARO+9598ZioUwBxnUWjbPsXcC53x8tm6jEv9RoETIW/AEC/9wON5q0McTbo69fuTpbpPIWAxB6IEgql/lOn1oP70xIJpXtmJAW7csH6VxvRpzbyiY02i9TPRR2jXCwdnnobb4x3IgtXqMpL2Nn2xVlsWteCkpqAk9JUZs0mMP9J4y+Sd7pc+16HPcOnM9vIJcP3aE5eKccfhuXTRv/7S4I8= ceph-2a682800-e989-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-24T02:51:39.269 INFO:teuthology.orchestra.run.smithi188.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4lFKfDQCQvBQuy3vj5h7NI+H7m+BxgALhKKgys5uJU/9U5Ih0DqN6SpoTFCu7I/4pOT19nOONPdtc51M0zWRqW9oDWIOJRAw1uSgo0Z1b/Gr5uH1H3vyF8GxBCSNEz6t5FI3KWcj/8Pka7ngQHosu5u3ucWazg/FoaPfCs9tzE1xhi5OBA5g53PIE6wkaUpYIlwqQfQz9LAnayND8J6ho2yeaB3YPtUZX5OQ5Ck+fZAaMH+2QxH7ed42Ih9NYqcO4SBHMgHqcQsqzpX9D+tPdShzRohbTjS2qhg+K4uWAARO+9598ZioUwBxnUWjbPsXcC53x8tm6jEv9RoETIW/AEC/9wON5q0McTbo69fuTpbpPIWAxB6IEgql/lOn1oP70xIJpXtmJAW7csH6VxvRpzbyiY02i9TPRR2jXCwdnnobb4x3IgtXqMpL2Nn2xVlsWteCkpqAk9JUZs0mMP9J4y+Sd7pc+16HPcOnM9vIJcP3aE5eKccfhuXTRv/7S4I8= ceph-2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:51:39.284 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-24T02:51:39.455 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:39 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/386745991' entity='client.admin' 2024-03-24T02:51:39.523 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:51:42.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.260+0000 7f0b23249700 1 -- 172.21.15.17:0/3549637969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b1c100ac0 msgr2=0x7f0b1c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:42.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.260+0000 7f0b23249700 1 --2- 172.21.15.17:0/3549637969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b1c100ac0 0x7f0b1c100ee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009910 tx=0x7f0b10009c20 comp rx=0 tx=0).stop 2024-03-24T02:51:42.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.261+0000 7f0b23249700 1 -- 172.21.15.17:0/3549637969 shutdown_connections 2024-03-24T02:51:42.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.261+0000 7f0b23249700 1 --2- 172.21.15.17:0/3549637969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b1c100ac0 0x7f0b1c100ee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:42.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.261+0000 7f0b23249700 1 -- 172.21.15.17:0/3549637969 >> 172.21.15.17:0/3549637969 conn(0x7f0b1c0fc060 msgr2=0x7f0b1c0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:42.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.261+0000 7f0b23249700 1 -- 172.21.15.17:0/3549637969 shutdown_connections 2024-03-24T02:51:42.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.261+0000 7f0b23249700 1 -- 172.21.15.17:0/3549637969 wait complete. 2024-03-24T02:51:42.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.261+0000 7f0b23249700 1 Processor -- start 2024-03-24T02:51:42.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.261+0000 7f0b23249700 1 -- start start 2024-03-24T02:51:42.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.261+0000 7f0b23249700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b1c100ac0 0x7f0b1c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:42.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.261+0000 7f0b23249700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c106b50 con 0x7f0b1c100ac0 2024-03-24T02:51:42.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.262+0000 7f0b20fe5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b1c100ac0 0x7f0b1c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:42.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.262+0000 7f0b20fe5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b1c100ac0 0x7f0b1c1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53948/0 (socket says 172.21.15.17:53948) 2024-03-24T02:51:42.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.262+0000 7f0b20fe5700 1 -- 172.21.15.17:0/1521140914 learned_addr learned my addr 172.21.15.17:0/1521140914 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:42.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.262+0000 7f0b20fe5700 1 -- 172.21.15.17:0/1521140914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c197800 con 0x7f0b1c100ac0 2024-03-24T02:51:42.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.262+0000 7f0b20fe5700 1 --2- 172.21.15.17:0/1521140914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b1c100ac0 0x7f0b1c1972c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005850 tx=0x7f0b10005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:42.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.262+0000 7f0b19ffb700 1 -- 172.21.15.17:0/1521140914 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b10026070 con 0x7f0b1c100ac0 2024-03-24T02:51:42.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.262+0000 7f0b19ffb700 1 -- 172.21.15.17:0/1521140914 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b100044a0 con 0x7f0b1c100ac0 2024-03-24T02:51:42.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.262+0000 7f0b19ffb700 1 -- 172.21.15.17:0/1521140914 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b10003ab0 con 0x7f0b1c100ac0 2024-03-24T02:51:42.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.262+0000 7f0b23249700 1 -- 172.21.15.17:0/1521140914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1c197a90 con 0x7f0b1c100ac0 2024-03-24T02:51:42.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.263+0000 7f0b23249700 1 -- 172.21.15.17:0/1521140914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1c19a780 con 0x7f0b1c100ac0 2024-03-24T02:51:42.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.264+0000 7f0b19ffb700 1 -- 172.21.15.17:0/1521140914 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f0b10029070 con 0x7f0b1c100ac0 2024-03-24T02:51:42.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.264+0000 7f0b23249700 1 -- 172.21.15.17:0/1521140914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1c1072d0 con 0x7f0b1c100ac0 2024-03-24T02:51:42.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.264+0000 7f0b19ffb700 1 --2- 172.21.15.17:0/1521140914 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f0b0c039020 0x7f0b0c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:42.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.264+0000 7f0b19ffb700 1 -- 172.21.15.17:0/1521140914 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0b1001e070 con 0x7f0b1c100ac0 2024-03-24T02:51:42.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.264+0000 7f0b1bfff700 1 --2- 172.21.15.17:0/1521140914 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f0b0c039020 0x7f0b0c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:42.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.264+0000 7f0b1bfff700 1 --2- 172.21.15.17:0/1521140914 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f0b0c039020 0x7f0b0c03b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b08006fd0 tx=0x7f0b08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:42.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.269+0000 7f0b19ffb700 1 -- 172.21.15.17:0/1521140914 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0b10013070 con 0x7f0b1c100ac0 2024-03-24T02:51:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.474+0000 7f0b23249700 1 -- 172.21.15.17:0/1521140914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f0b1c02db40 con 0x7f0b1c100ac0 2024-03-24T02:51:42.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.476+0000 7f0b19ffb700 1 -- 172.21.15.17:0/1521140914 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0b10030020 con 0x7f0b1c100ac0 2024-03-24T02:51:42.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.478+0000 7f0b23249700 1 -- 172.21.15.17:0/1521140914 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f0b0c039020 msgr2=0x7f0b0c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:42.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.479+0000 7f0b23249700 1 --2- 172.21.15.17:0/1521140914 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f0b0c039020 0x7f0b0c03b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b08006fd0 tx=0x7f0b08006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:42.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.479+0000 7f0b23249700 1 -- 172.21.15.17:0/1521140914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b1c100ac0 msgr2=0x7f0b1c1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:42.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.479+0000 7f0b23249700 1 --2- 172.21.15.17:0/1521140914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b1c100ac0 0x7f0b1c1972c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005850 tx=0x7f0b10005030 comp rx=0 tx=0).stop 2024-03-24T02:51:42.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.479+0000 7f0b23249700 1 -- 172.21.15.17:0/1521140914 shutdown_connections 2024-03-24T02:51:42.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.479+0000 7f0b23249700 1 --2- 172.21.15.17:0/1521140914 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f0b0c039020 0x7f0b0c03b4e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:42.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.479+0000 7f0b23249700 1 --2- 172.21.15.17:0/1521140914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b1c100ac0 0x7f0b1c1972c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:42.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.479+0000 7f0b23249700 1 -- 172.21.15.17:0/1521140914 >> 172.21.15.17:0/1521140914 conn(0x7f0b1c0fc060 msgr2=0x7f0b1c0699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:42.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.479+0000 7f0b23249700 1 -- 172.21.15.17:0/1521140914 shutdown_connections 2024-03-24T02:51:42.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:42.479+0000 7f0b23249700 1 -- 172.21.15.17:0/1521140914 wait complete. 2024-03-24T02:51:43.106 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-24T02:51:43.106 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-24T02:51:43.683 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:51:43.805 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:43 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1521140914' entity='client.admin' 2024-03-24T02:51:44.820 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:44.820 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:44.820 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi017", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:51:44.820 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:44.820 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:51:44.820 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:45.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.335+0000 7f49fe3fd700 1 -- 172.21.15.17:0/2646285417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80ffeb0 msgr2=0x7f49f81002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.335+0000 7f49fe3fd700 1 --2- 172.21.15.17:0/2646285417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80ffeb0 0x7f49f81002d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009910 tx=0x7f49e8009c20 comp rx=0 tx=0).stop 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.335+0000 7f49fe3fd700 1 -- 172.21.15.17:0/2646285417 shutdown_connections 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.335+0000 7f49fe3fd700 1 --2- 172.21.15.17:0/2646285417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80ffeb0 0x7f49f81002d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.335+0000 7f49fe3fd700 1 -- 172.21.15.17:0/2646285417 >> 172.21.15.17:0/2646285417 conn(0x7f49f80fb810 msgr2=0x7f49f80fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.335+0000 7f49fe3fd700 1 -- 172.21.15.17:0/2646285417 shutdown_connections 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.335+0000 7f49fe3fd700 1 -- 172.21.15.17:0/2646285417 wait complete. 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.335+0000 7f49fe3fd700 1 Processor -- start 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.335+0000 7f49fe3fd700 1 -- start start 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.336+0000 7f49fe3fd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80ffeb0 0x7f49f81994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.336+0000 7f49fe3fd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f8107c80 con 0x7f49f80ffeb0 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.336+0000 7f49f7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80ffeb0 0x7f49f81994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.336+0000 7f49f7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80ffeb0 0x7f49f81994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53966/0 (socket says 172.21.15.17:53966) 2024-03-24T02:51:45.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.336+0000 7f49f7fff700 1 -- 172.21.15.17:0/1766513192 learned_addr learned my addr 172.21.15.17:0/1766513192 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:45.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.336+0000 7f49f7fff700 1 -- 172.21.15.17:0/1766513192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f8199a30 con 0x7f49f80ffeb0 2024-03-24T02:51:45.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.336+0000 7f49f7fff700 1 --2- 172.21.15.17:0/1766513192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80ffeb0 0x7f49f81994f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005850 tx=0x7f49e8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:45.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.336+0000 7f49f57fa700 1 -- 172.21.15.17:0/1766513192 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49e8026070 con 0x7f49f80ffeb0 2024-03-24T02:51:45.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.336+0000 7f49f57fa700 1 -- 172.21.15.17:0/1766513192 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49e80044a0 con 0x7f49f80ffeb0 2024-03-24T02:51:45.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.336+0000 7f49fe3fd700 1 -- 172.21.15.17:0/1766513192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f8199cc0 con 0x7f49f80ffeb0 2024-03-24T02:51:45.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.336+0000 7f49f57fa700 1 -- 172.21.15.17:0/1766513192 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49e8003ab0 con 0x7f49f80ffeb0 2024-03-24T02:51:45.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.336+0000 7f49fe3fd700 1 -- 172.21.15.17:0/1766513192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f819c9b0 con 0x7f49f80ffeb0 2024-03-24T02:51:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.337+0000 7f49f57fa700 1 -- 172.21.15.17:0/1766513192 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f49e8029070 con 0x7f49f80ffeb0 2024-03-24T02:51:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.337+0000 7f49fe3fd700 1 -- 172.21.15.17:0/1766513192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f8108400 con 0x7f49f80ffeb0 2024-03-24T02:51:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.337+0000 7f49f57fa700 1 --2- 172.21.15.17:0/1766513192 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f49e0039020 0x7f49e003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.337+0000 7f49f57fa700 1 -- 172.21.15.17:0/1766513192 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f49e801e070 con 0x7f49f80ffeb0 2024-03-24T02:51:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.338+0000 7f49f77fe700 1 --2- 172.21.15.17:0/1766513192 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f49e0039020 0x7f49e003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.338+0000 7f49f77fe700 1 --2- 172.21.15.17:0/1766513192 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f49e0039020 0x7f49e003b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f49ec006fd0 tx=0x7f49ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:45.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.342+0000 7f49f57fa700 1 -- 172.21.15.17:0/1766513192 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f49e8013070 con 0x7f49f80ffeb0 2024-03-24T02:51:45.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.552+0000 7f49fe3fd700 1 -- 172.21.15.17:0/1766513192 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f49f802db40 con 0x7f49e0039020 2024-03-24T02:51:45.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.555+0000 7f49f57fa700 1 -- 172.21.15.17:0/1766513192 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f49f802db40 con 0x7f49e0039020 2024-03-24T02:51:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.557+0000 7f49fe3fd700 1 -- 172.21.15.17:0/1766513192 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f49e0039020 msgr2=0x7f49e003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.557+0000 7f49fe3fd700 1 --2- 172.21.15.17:0/1766513192 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f49e0039020 0x7f49e003b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f49ec006fd0 tx=0x7f49ec006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.557+0000 7f49fe3fd700 1 -- 172.21.15.17:0/1766513192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80ffeb0 msgr2=0x7f49f81994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.557+0000 7f49fe3fd700 1 --2- 172.21.15.17:0/1766513192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80ffeb0 0x7f49f81994f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005850 tx=0x7f49e8005030 comp rx=0 tx=0).stop 2024-03-24T02:51:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.557+0000 7f49fe3fd700 1 -- 172.21.15.17:0/1766513192 shutdown_connections 2024-03-24T02:51:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.557+0000 7f49fe3fd700 1 --2- 172.21.15.17:0/1766513192 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f49e0039020 0x7f49e003b4e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.557+0000 7f49fe3fd700 1 --2- 172.21.15.17:0/1766513192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49f80ffeb0 0x7f49f81994f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.557+0000 7f49fe3fd700 1 -- 172.21.15.17:0/1766513192 >> 172.21.15.17:0/1766513192 conn(0x7f49f80fb810 msgr2=0x7f49f80fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.557+0000 7f49fe3fd700 1 -- 172.21.15.17:0/1766513192 shutdown_connections 2024-03-24T02:51:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:45.557+0000 7f49fe3fd700 1 -- 172.21.15.17:0/1766513192 wait complete. 2024-03-24T02:51:46.507 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi019 2024-03-24T02:51:46.507 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-24T02:51:46.507 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/etc/ceph/ceph.conf 2024-03-24T02:51:46.526 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-24T02:51:46.526 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T02:51:46.585 INFO:tasks.cephadm:Adding host smithi019 to orchestrator... 2024-03-24T02:51:46.585 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch host add smithi019 2024-03-24T02:51:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:51:46.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:46 smithi017 ceph-mon[31926]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:51:46.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:46 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:46.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:46 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:51:46.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:46 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:51:46.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:46 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:51:46.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:46 smithi017 ceph-mon[31926]: Updating smithi017:/etc/ceph/ceph.conf 2024-03-24T02:51:46.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:46 smithi017 ceph-mon[31926]: Updating smithi017:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:51:46.941 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:46 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:46.941 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:46 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:46.941 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:46 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:47.885 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:47 smithi017 ceph-mon[31926]: Updating smithi017:/etc/ceph/ceph.client.admin.keyring 2024-03-24T02:51:47.885 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:47 smithi017 ceph-mon[31926]: Updating smithi017:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T02:51:48.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.404+0000 7f60c77d0700 1 -- 172.21.15.17:0/3882967064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60c0100450 msgr2=0x7f60c0100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:48.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.404+0000 7f60c77d0700 1 --2- 172.21.15.17:0/3882967064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60c0100450 0x7f60c0100830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009910 tx=0x7f60bc009c20 comp rx=0 tx=0).stop 2024-03-24T02:51:48.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.404+0000 7f60c77d0700 1 -- 172.21.15.17:0/3882967064 shutdown_connections 2024-03-24T02:51:48.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.404+0000 7f60c77d0700 1 --2- 172.21.15.17:0/3882967064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60c0100450 0x7f60c0100830 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:48.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.404+0000 7f60c77d0700 1 -- 172.21.15.17:0/3882967064 >> 172.21.15.17:0/3882967064 conn(0x7f60c00fbce0 msgr2=0x7f60c00fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:48.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.405+0000 7f60c77d0700 1 -- 172.21.15.17:0/3882967064 shutdown_connections 2024-03-24T02:51:48.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.405+0000 7f60c77d0700 1 -- 172.21.15.17:0/3882967064 wait complete. 2024-03-24T02:51:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.405+0000 7f60c77d0700 1 Processor -- start 2024-03-24T02:51:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.405+0000 7f60c77d0700 1 -- start start 2024-03-24T02:51:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.406+0000 7f60c77d0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60c0100450 0x7f60c01976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.406+0000 7f60c77d0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c010a820 con 0x7f60c0100450 2024-03-24T02:51:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.406+0000 7f60c556c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60c0100450 0x7f60c01976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.406+0000 7f60c556c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60c0100450 0x7f60c01976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53988/0 (socket says 172.21.15.17:53988) 2024-03-24T02:51:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.406+0000 7f60c556c700 1 -- 172.21.15.17:0/332352437 learned_addr learned my addr 172.21.15.17:0/332352437 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.406+0000 7f60c556c700 1 -- 172.21.15.17:0/332352437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c0197be0 con 0x7f60c0100450 2024-03-24T02:51:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.406+0000 7f60c556c700 1 --2- 172.21.15.17:0/332352437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60c0100450 0x7f60c01976a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f60bc005850 tx=0x7f60bc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.406+0000 7f60b67fc700 1 -- 172.21.15.17:0/332352437 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60bc026070 con 0x7f60c0100450 2024-03-24T02:51:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.406+0000 7f60b67fc700 1 -- 172.21.15.17:0/332352437 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60bc00bbd0 con 0x7f60c0100450 2024-03-24T02:51:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.406+0000 7f60c77d0700 1 -- 172.21.15.17:0/332352437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c0197ed0 con 0x7f60c0100450 2024-03-24T02:51:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.406+0000 7f60b67fc700 1 -- 172.21.15.17:0/332352437 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60bc003970 con 0x7f60c0100450 2024-03-24T02:51:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.406+0000 7f60c77d0700 1 -- 172.21.15.17:0/332352437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c019bf70 con 0x7f60c0100450 2024-03-24T02:51:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.407+0000 7f60b67fc700 1 -- 172.21.15.17:0/332352437 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f60bc029070 con 0x7f60c0100450 2024-03-24T02:51:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.407+0000 7f60c77d0700 1 -- 172.21.15.17:0/332352437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c0101930 con 0x7f60c0100450 2024-03-24T02:51:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.407+0000 7f60b67fc700 1 --2- 172.21.15.17:0/332352437 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f60ac039020 0x7f60ac03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.407+0000 7f60b67fc700 1 -- 172.21.15.17:0/332352437 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f60bc01e070 con 0x7f60c0100450 2024-03-24T02:51:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.408+0000 7f60c4d6b700 1 --2- 172.21.15.17:0/332352437 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f60ac039020 0x7f60ac03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.408+0000 7f60c4d6b700 1 --2- 172.21.15.17:0/332352437 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f60ac039020 0x7f60ac03b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f60b0006fd0 tx=0x7f60b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.412+0000 7f60b67fc700 1 -- 172.21.15.17:0/332352437 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f60bc013070 con 0x7f60c0100450 2024-03-24T02:51:48.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:48.621+0000 7f60c77d0700 1 -- 172.21.15.17:0/332352437 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "target": ["mon-mgr", ""]}) v1 -- 0x7f60c0198a00 con 0x7f60ac039020 2024-03-24T02:51:49.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:48 smithi017 ceph-mon[31926]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:51:50.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:49 smithi017 ceph-mon[31926]: Deploying cephadm binary to smithi019 2024-03-24T02:51:50.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:49 smithi017 ceph-mon[31926]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:51:52.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:51 smithi017 ceph-mon[31926]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:51:53.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:53.048+0000 7f60b67fc700 1 -- 172.21.15.17:0/332352437 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f60c0198a00 con 0x7f60ac039020 2024-03-24T02:51:53.049 INFO:teuthology.orchestra.run.smithi017.stdout:Added host 'smithi019' with addr '172.21.15.19' 2024-03-24T02:51:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:53.051+0000 7f60c77d0700 1 -- 172.21.15.17:0/332352437 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f60ac039020 msgr2=0x7f60ac03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:53.051+0000 7f60c77d0700 1 --2- 172.21.15.17:0/332352437 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f60ac039020 0x7f60ac03b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f60b0006fd0 tx=0x7f60b0006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:53.051+0000 7f60c77d0700 1 -- 172.21.15.17:0/332352437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60c0100450 msgr2=0x7f60c01976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:53.051+0000 7f60c77d0700 1 --2- 172.21.15.17:0/332352437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60c0100450 0x7f60c01976a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f60bc005850 tx=0x7f60bc005030 comp rx=0 tx=0).stop 2024-03-24T02:51:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:53.051+0000 7f60c77d0700 1 -- 172.21.15.17:0/332352437 shutdown_connections 2024-03-24T02:51:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:53.051+0000 7f60c77d0700 1 --2- 172.21.15.17:0/332352437 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f60ac039020 0x7f60ac03b4e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:53.051+0000 7f60c77d0700 1 --2- 172.21.15.17:0/332352437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60c0100450 0x7f60c01976a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:53.051+0000 7f60c77d0700 1 -- 172.21.15.17:0/332352437 >> 172.21.15.17:0/332352437 conn(0x7f60c00fbce0 msgr2=0x7f60c0108500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:53.051+0000 7f60c77d0700 1 -- 172.21.15.17:0/332352437 shutdown_connections 2024-03-24T02:51:53.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:53.051+0000 7f60c77d0700 1 -- 172.21.15.17:0/332352437 wait complete. 2024-03-24T02:51:53.658 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-24T02:51:53.903 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:51:54.352 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:54 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:54.352 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:54 smithi017 ceph-mon[31926]: Added host smithi019 2024-03-24T02:51:54.352 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:54 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:51:54.352 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:54 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:54.352 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:54 smithi017 ceph-mon[31926]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:51:55.254 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:55 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.510+0000 7f4af33fb700 1 -- 172.21.15.17:0/3921493254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aec1002c0 msgr2=0x7f4aec1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.510+0000 7f4af33fb700 1 --2- 172.21.15.17:0/3921493254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aec1002c0 0x7f4aec1006a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009910 tx=0x7f4adc009c20 comp rx=0 tx=0).stop 2024-03-24T02:51:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.510+0000 7f4af33fb700 1 -- 172.21.15.17:0/3921493254 shutdown_connections 2024-03-24T02:51:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.510+0000 7f4af33fb700 1 --2- 172.21.15.17:0/3921493254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aec1002c0 0x7f4aec1006a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.510+0000 7f4af33fb700 1 -- 172.21.15.17:0/3921493254 >> 172.21.15.17:0/3921493254 conn(0x7f4aec0fbce0 msgr2=0x7f4aec0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.511+0000 7f4af33fb700 1 -- 172.21.15.17:0/3921493254 shutdown_connections 2024-03-24T02:51:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.511+0000 7f4af33fb700 1 -- 172.21.15.17:0/3921493254 wait complete. 2024-03-24T02:51:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.511+0000 7f4af33fb700 1 Processor -- start 2024-03-24T02:51:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.511+0000 7f4af33fb700 1 -- start start 2024-03-24T02:51:55.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.511+0000 7f4af33fb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aec1002c0 0x7f4aec19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:55.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.511+0000 7f4af33fb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec10c1b0 con 0x7f4aec1002c0 2024-03-24T02:51:55.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.512+0000 7f4af1197700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aec1002c0 0x7f4aec19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:55.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.512+0000 7f4af1197700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aec1002c0 0x7f4aec19f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42914/0 (socket says 172.21.15.17:42914) 2024-03-24T02:51:55.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.512+0000 7f4af1197700 1 -- 172.21.15.17:0/875106693 learned_addr learned my addr 172.21.15.17:0/875106693 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:55.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.512+0000 7f4af1197700 1 -- 172.21.15.17:0/875106693 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec19fae0 con 0x7f4aec1002c0 2024-03-24T02:51:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.512+0000 7f4af1197700 1 --2- 172.21.15.17:0/875106693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aec1002c0 0x7f4aec19f5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4adc006010 tx=0x7f4adc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.512+0000 7f4ae27fc700 1 -- 172.21.15.17:0/875106693 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4adc026070 con 0x7f4aec1002c0 2024-03-24T02:51:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.512+0000 7f4af33fb700 1 -- 172.21.15.17:0/875106693 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec19fdd0 con 0x7f4aec1002c0 2024-03-24T02:51:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.512+0000 7f4ae27fc700 1 -- 172.21.15.17:0/875106693 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4adc00bbd0 con 0x7f4aec1002c0 2024-03-24T02:51:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.512+0000 7f4ae27fc700 1 -- 172.21.15.17:0/875106693 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4adc003910 con 0x7f4aec1002c0 2024-03-24T02:51:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.512+0000 7f4af33fb700 1 -- 172.21.15.17:0/875106693 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aec199840 con 0x7f4aec1002c0 2024-03-24T02:51:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.513+0000 7f4ae27fc700 1 -- 172.21.15.17:0/875106693 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f4adc029070 con 0x7f4aec1002c0 2024-03-24T02:51:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.513+0000 7f4ae27fc700 1 --2- 172.21.15.17:0/875106693 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4ad8038fc0 0x7f4ad803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.513+0000 7f4ae27fc700 1 -- 172.21.15.17:0/875106693 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4adc01e070 con 0x7f4aec1002c0 2024-03-24T02:51:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.513+0000 7f4af33fb700 1 -- 172.21.15.17:0/875106693 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aec103380 con 0x7f4aec1002c0 2024-03-24T02:51:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.513+0000 7f4af0996700 1 --2- 172.21.15.17:0/875106693 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4ad8038fc0 0x7f4ad803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.514+0000 7f4af0996700 1 --2- 172.21.15.17:0/875106693 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4ad8038fc0 0x7f4ad803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8006fd0 tx=0x7f4ae8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:55.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.518+0000 7f4ae27fc700 1 -- 172.21.15.17:0/875106693 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4adc030070 con 0x7f4aec1002c0 2024-03-24T02:51:55.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.728+0000 7f4af33fb700 1 -- 172.21.15.17:0/875106693 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4aec0008d0 con 0x7f4ad8038fc0 2024-03-24T02:51:55.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.729+0000 7f4ae27fc700 1 -- 172.21.15.17:0/875106693 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f4aec0008d0 con 0x7f4ad8038fc0 2024-03-24T02:51:55.729 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:51:55.729 INFO:teuthology.orchestra.run.smithi017.stdout:[{"addr": "172.21.15.17", "hostname": "smithi017", "labels": [], "status": ""}, {"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}] 2024-03-24T02:51:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.731+0000 7f4af33fb700 1 -- 172.21.15.17:0/875106693 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4ad8038fc0 msgr2=0x7f4ad803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.731+0000 7f4af33fb700 1 --2- 172.21.15.17:0/875106693 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4ad8038fc0 0x7f4ad803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8006fd0 tx=0x7f4ae8006e40 comp rx=0 tx=0).stop 2024-03-24T02:51:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.731+0000 7f4af33fb700 1 -- 172.21.15.17:0/875106693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aec1002c0 msgr2=0x7f4aec19f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.731+0000 7f4af33fb700 1 --2- 172.21.15.17:0/875106693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aec1002c0 0x7f4aec19f5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4adc006010 tx=0x7f4adc005030 comp rx=0 tx=0).stop 2024-03-24T02:51:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.731+0000 7f4af33fb700 1 -- 172.21.15.17:0/875106693 shutdown_connections 2024-03-24T02:51:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.731+0000 7f4af33fb700 1 --2- 172.21.15.17:0/875106693 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4ad8038fc0 0x7f4ad803b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.731+0000 7f4af33fb700 1 --2- 172.21.15.17:0/875106693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aec1002c0 0x7f4aec19f5a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.731+0000 7f4af33fb700 1 -- 172.21.15.17:0/875106693 >> 172.21.15.17:0/875106693 conn(0x7f4aec0fbce0 msgr2=0x7f4aec102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.731+0000 7f4af33fb700 1 -- 172.21.15.17:0/875106693 shutdown_connections 2024-03-24T02:51:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:55.731+0000 7f4af33fb700 1 -- 172.21.15.17:0/875106693 wait complete. 2024-03-24T02:51:56.389 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi188 2024-03-24T02:51:56.389 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-24T02:51:56.389 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.conf 2024-03-24T02:51:56.408 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-24T02:51:56.408 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T02:51:56.465 INFO:tasks.cephadm:Adding host smithi188 to orchestrator... 2024-03-24T02:51:56.465 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch host add smithi188 2024-03-24T02:51:56.708 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:51:57.038 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:56 smithi017 ceph-mon[31926]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T02:51:57.038 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:56 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:51:57.038 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:56 smithi017 ceph-mon[31926]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:51:58.194 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:57 smithi017 ceph-mon[31926]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:51:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.217+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/406603866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0f4102d10 msgr2=0x7fb0f41030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:51:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.217+0000 7fb0fa2e5700 1 --2- 172.21.15.17:0/406603866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0f4102d10 0x7fb0f41030f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009910 tx=0x7fb0e4009c20 comp rx=0 tx=0).stop 2024-03-24T02:51:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.217+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/406603866 shutdown_connections 2024-03-24T02:51:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.217+0000 7fb0fa2e5700 1 --2- 172.21.15.17:0/406603866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0f4102d10 0x7fb0f41030f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:51:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.217+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/406603866 >> 172.21.15.17:0/406603866 conn(0x7fb0f40fc540 msgr2=0x7fb0f40fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:51:58.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.217+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/406603866 shutdown_connections 2024-03-24T02:51:58.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.217+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/406603866 wait complete. 2024-03-24T02:51:58.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.217+0000 7fb0fa2e5700 1 Processor -- start 2024-03-24T02:51:58.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.217+0000 7fb0fa2e5700 1 -- start start 2024-03-24T02:51:58.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.218+0000 7fb0fa2e5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0f4102d10 0x7fb0f4100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:58.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.218+0000 7fb0fa2e5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f410c200 con 0x7fb0f4102d10 2024-03-24T02:51:58.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.218+0000 7fb0f3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0f4102d10 0x7fb0f4100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:58.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.218+0000 7fb0f3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0f4102d10 0x7fb0f4100970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42938/0 (socket says 172.21.15.17:42938) 2024-03-24T02:51:58.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.218+0000 7fb0f3fff700 1 -- 172.21.15.17:0/1280333212 learned_addr learned my addr 172.21.15.17:0/1280333212 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:51:58.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.218+0000 7fb0f3fff700 1 -- 172.21.15.17:0/1280333212 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f4102860 con 0x7fb0f4102d10 2024-03-24T02:51:58.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.218+0000 7fb0f3fff700 1 --2- 172.21.15.17:0/1280333212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0f4102d10 0x7fb0f4100970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4006010 tx=0x7fb0e4005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:58.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.218+0000 7fb0f17fa700 1 -- 172.21.15.17:0/1280333212 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0e4026070 con 0x7fb0f4102d10 2024-03-24T02:51:58.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.219+0000 7fb0f17fa700 1 -- 172.21.15.17:0/1280333212 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0e400bc30 con 0x7fb0f4102d10 2024-03-24T02:51:58.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.219+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/1280333212 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f4100eb0 con 0x7fb0f4102d10 2024-03-24T02:51:58.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.219+0000 7fb0f17fa700 1 -- 172.21.15.17:0/1280333212 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0e4003ab0 con 0x7fb0f4102d10 2024-03-24T02:51:58.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.219+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/1280333212 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f4101370 con 0x7fb0f4102d10 2024-03-24T02:51:58.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.219+0000 7fb0f17fa700 1 -- 172.21.15.17:0/1280333212 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fb0e4029070 con 0x7fb0f4102d10 2024-03-24T02:51:58.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.220+0000 7fb0f17fa700 1 --2- 172.21.15.17:0/1280333212 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb0e0039020 0x7fb0e003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:51:58.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.220+0000 7fb0f17fa700 1 -- 172.21.15.17:0/1280333212 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb0e401e070 con 0x7fb0f4102d10 2024-03-24T02:51:58.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.220+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/1280333212 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f406a6f0 con 0x7fb0f4102d10 2024-03-24T02:51:58.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.220+0000 7fb0f37fe700 1 --2- 172.21.15.17:0/1280333212 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb0e0039020 0x7fb0e003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:51:58.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.220+0000 7fb0f37fe700 1 --2- 172.21.15.17:0/1280333212 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb0e0039020 0x7fb0e003b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc006fd0 tx=0x7fb0dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:51:58.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.225+0000 7fb0f17fa700 1 -- 172.21.15.17:0/1280333212 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb0e4030020 con 0x7fb0f4102d10 2024-03-24T02:51:58.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:51:58.435+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/1280333212 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0f4101ea0 con 0x7fb0e0039020 2024-03-24T02:52:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:59 smithi017 ceph-mon[31926]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:52:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:52:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:51:59 smithi017 ceph-mon[31926]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-24T02:52:01.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:00 smithi017 ceph-mon[31926]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:01.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:00 smithi017 ceph-mon[31926]: Deploying cephadm binary to smithi188 2024-03-24T02:52:01.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:00 smithi017 ceph-mon[31926]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-24T02:52:01.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:00 smithi017 ceph-mon[31926]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T02:52:01.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:00 smithi017 ceph-mon[31926]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:02.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:01 smithi017 ceph-mon[31926]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:02.732 INFO:teuthology.orchestra.run.smithi017.stdout:Added host 'smithi188' with addr '172.21.15.188' 2024-03-24T02:52:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:02.731+0000 7fb0f17fa700 1 -- 172.21.15.17:0/1280333212 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb0f4101ea0 con 0x7fb0e0039020 2024-03-24T02:52:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:02.733+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/1280333212 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb0e0039020 msgr2=0x7fb0e003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:02.733+0000 7fb0fa2e5700 1 --2- 172.21.15.17:0/1280333212 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb0e0039020 0x7fb0e003b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc006fd0 tx=0x7fb0dc006e40 comp rx=0 tx=0).stop 2024-03-24T02:52:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:02.733+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/1280333212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0f4102d10 msgr2=0x7fb0f4100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:02.733+0000 7fb0fa2e5700 1 --2- 172.21.15.17:0/1280333212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0f4102d10 0x7fb0f4100970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4006010 tx=0x7fb0e4005050 comp rx=0 tx=0).stop 2024-03-24T02:52:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:02.734+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/1280333212 shutdown_connections 2024-03-24T02:52:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:02.734+0000 7fb0fa2e5700 1 --2- 172.21.15.17:0/1280333212 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb0e0039020 0x7fb0e003b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:02.734+0000 7fb0fa2e5700 1 --2- 172.21.15.17:0/1280333212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0f4102d10 0x7fb0f4100970 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:02.734+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/1280333212 >> 172.21.15.17:0/1280333212 conn(0x7fb0f40fc540 msgr2=0x7fb0f4103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:02.734+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/1280333212 shutdown_connections 2024-03-24T02:52:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:02.734+0000 7fb0fa2e5700 1 -- 172.21.15.17:0/1280333212 wait complete. 2024-03-24T02:52:03.372 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:03.372 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:03.373 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:03.373 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:03.373 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:03 smithi017 ceph-mon[31926]: Added host smithi188 2024-03-24T02:52:03.373 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:03.404 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-24T02:52:03.649 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:52:04.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:04 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:04.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:04 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:04.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:04 smithi017 ceph-mon[31926]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:05.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.134+0000 7fdbee505700 1 -- 172.21.15.17:0/620465141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe8106c30 msgr2=0x7fdbe8107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:05.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.134+0000 7fdbee505700 1 --2- 172.21.15.17:0/620465141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe8106c30 0x7fdbe8107010 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009910 tx=0x7fdbd8009c20 comp rx=0 tx=0).stop 2024-03-24T02:52:05.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.134+0000 7fdbee505700 1 -- 172.21.15.17:0/620465141 shutdown_connections 2024-03-24T02:52:05.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.134+0000 7fdbee505700 1 --2- 172.21.15.17:0/620465141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe8106c30 0x7fdbe8107010 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:05.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.134+0000 7fdbee505700 1 -- 172.21.15.17:0/620465141 >> 172.21.15.17:0/620465141 conn(0x7fdbe80fbd10 msgr2=0x7fdbe80fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:05.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.134+0000 7fdbee505700 1 -- 172.21.15.17:0/620465141 shutdown_connections 2024-03-24T02:52:05.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.134+0000 7fdbee505700 1 -- 172.21.15.17:0/620465141 wait complete. 2024-03-24T02:52:05.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.134+0000 7fdbee505700 1 Processor -- start 2024-03-24T02:52:05.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.134+0000 7fdbee505700 1 -- start start 2024-03-24T02:52:05.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.135+0000 7fdbee505700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe8106c30 0x7fdbe8196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:05.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.135+0000 7fdbee505700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe810b230 con 0x7fdbe8106c30 2024-03-24T02:52:05.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.135+0000 7fdbe7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe8106c30 0x7fdbe8196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:05.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.135+0000 7fdbe7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe8106c30 0x7fdbe8196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46326/0 (socket says 172.21.15.17:46326) 2024-03-24T02:52:05.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.135+0000 7fdbe7fff700 1 -- 172.21.15.17:0/2480661374 learned_addr learned my addr 172.21.15.17:0/2480661374 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:52:05.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.135+0000 7fdbe7fff700 1 -- 172.21.15.17:0/2480661374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe8196b70 con 0x7fdbe8106c30 2024-03-24T02:52:05.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.135+0000 7fdbe7fff700 1 --2- 172.21.15.17:0/2480661374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe8106c30 0x7fdbe8196630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8005850 tx=0x7fdbd8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:05.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.136+0000 7fdbe57fa700 1 -- 172.21.15.17:0/2480661374 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbd8026070 con 0x7fdbe8106c30 2024-03-24T02:52:05.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.136+0000 7fdbee505700 1 -- 172.21.15.17:0/2480661374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe8196e00 con 0x7fdbe8106c30 2024-03-24T02:52:05.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.136+0000 7fdbe57fa700 1 -- 172.21.15.17:0/2480661374 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdbd800bc30 con 0x7fdbe8106c30 2024-03-24T02:52:05.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.136+0000 7fdbe57fa700 1 -- 172.21.15.17:0/2480661374 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbd8003a20 con 0x7fdbe8106c30 2024-03-24T02:52:05.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.136+0000 7fdbee505700 1 -- 172.21.15.17:0/2480661374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe8197260 con 0x7fdbe8106c30 2024-03-24T02:52:05.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.137+0000 7fdbe57fa700 1 -- 172.21.15.17:0/2480661374 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fdbd8029070 con 0x7fdbe8106c30 2024-03-24T02:52:05.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.137+0000 7fdbe57fa700 1 --2- 172.21.15.17:0/2480661374 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdbd4038fc0 0x7fdbd403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:05.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.137+0000 7fdbee505700 1 -- 172.21.15.17:0/2480661374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe81023c0 con 0x7fdbe8106c30 2024-03-24T02:52:05.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.137+0000 7fdbe57fa700 1 -- 172.21.15.17:0/2480661374 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdbd801e070 con 0x7fdbe8106c30 2024-03-24T02:52:05.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.137+0000 7fdbe77fe700 1 --2- 172.21.15.17:0/2480661374 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdbd4038fc0 0x7fdbd403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:05.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.137+0000 7fdbe77fe700 1 --2- 172.21.15.17:0/2480661374 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdbd4038fc0 0x7fdbd403b480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0006fd0 tx=0x7fdbd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:05.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.142+0000 7fdbe57fa700 1 -- 172.21.15.17:0/2480661374 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdbd8013070 con 0x7fdbe8106c30 2024-03-24T02:52:05.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.355+0000 7fdbee505700 1 -- 172.21.15.17:0/2480661374 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdbe8197990 con 0x7fdbd4038fc0 2024-03-24T02:52:05.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.356+0000 7fdbe57fa700 1 -- 172.21.15.17:0/2480661374 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7fdbe8197990 con 0x7fdbd4038fc0 2024-03-24T02:52:05.356 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:52:05.356 INFO:teuthology.orchestra.run.smithi017.stdout:[{"addr": "172.21.15.17", "hostname": "smithi017", "labels": [], "status": ""}, {"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.188", "hostname": "smithi188", "labels": [], "status": ""}] 2024-03-24T02:52:05.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.358+0000 7fdbee505700 1 -- 172.21.15.17:0/2480661374 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdbd4038fc0 msgr2=0x7fdbd403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:05.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.358+0000 7fdbee505700 1 --2- 172.21.15.17:0/2480661374 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdbd4038fc0 0x7fdbd403b480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0006fd0 tx=0x7fdbd0006e40 comp rx=0 tx=0).stop 2024-03-24T02:52:05.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.358+0000 7fdbee505700 1 -- 172.21.15.17:0/2480661374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe8106c30 msgr2=0x7fdbe8196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:05.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.358+0000 7fdbee505700 1 --2- 172.21.15.17:0/2480661374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe8106c30 0x7fdbe8196630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8005850 tx=0x7fdbd8005030 comp rx=0 tx=0).stop 2024-03-24T02:52:05.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.358+0000 7fdbee505700 1 -- 172.21.15.17:0/2480661374 shutdown_connections 2024-03-24T02:52:05.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.358+0000 7fdbee505700 1 --2- 172.21.15.17:0/2480661374 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdbd4038fc0 0x7fdbd403b480 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:05.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.358+0000 7fdbee505700 1 --2- 172.21.15.17:0/2480661374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe8106c30 0x7fdbe8196630 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:05.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.358+0000 7fdbee505700 1 -- 172.21.15.17:0/2480661374 >> 172.21.15.17:0/2480661374 conn(0x7fdbe80fbd10 msgr2=0x7fdbe81084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:05.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.358+0000 7fdbee505700 1 -- 172.21.15.17:0/2480661374 shutdown_connections 2024-03-24T02:52:05.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:05.358+0000 7fdbee505700 1 -- 172.21.15.17:0/2480661374 wait complete. 2024-03-24T02:52:05.945 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-24T02:52:05.945 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-24T02:52:06.181 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:52:06.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:06.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:06 smithi017 ceph-mon[31926]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T02:52:06.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:06 smithi017 ceph-mon[31926]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:07.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.658+0000 7f40d42c4700 1 -- 172.21.15.17:0/2875775991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40cc101dd0 msgr2=0x7f40cc1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:07.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.658+0000 7f40d42c4700 1 --2- 172.21.15.17:0/2875775991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40cc101dd0 0x7f40cc1021b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009910 tx=0x7f40c8009c20 comp rx=0 tx=0).stop 2024-03-24T02:52:07.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.658+0000 7f40d42c4700 1 -- 172.21.15.17:0/2875775991 shutdown_connections 2024-03-24T02:52:07.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.658+0000 7f40d42c4700 1 --2- 172.21.15.17:0/2875775991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40cc101dd0 0x7f40cc1021b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:07.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.658+0000 7f40d42c4700 1 -- 172.21.15.17:0/2875775991 >> 172.21.15.17:0/2875775991 conn(0x7f40cc0fd680 msgr2=0x7f40cc0ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:07.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.658+0000 7f40d42c4700 1 -- 172.21.15.17:0/2875775991 shutdown_connections 2024-03-24T02:52:07.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.658+0000 7f40d42c4700 1 -- 172.21.15.17:0/2875775991 wait complete. 2024-03-24T02:52:07.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.659+0000 7f40d42c4700 1 Processor -- start 2024-03-24T02:52:07.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.659+0000 7f40d42c4700 1 -- start start 2024-03-24T02:52:07.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.659+0000 7f40d42c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40cc101dd0 0x7f40cc074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:07.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.659+0000 7f40d42c4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc10c1a0 con 0x7f40cc101dd0 2024-03-24T02:52:07.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.659+0000 7f40d2060700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40cc101dd0 0x7f40cc074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.659+0000 7f40d2060700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40cc101dd0 0x7f40cc074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46346/0 (socket says 172.21.15.17:46346) 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.659+0000 7f40d2060700 1 -- 172.21.15.17:0/1994233701 learned_addr learned my addr 172.21.15.17:0/1994233701 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.659+0000 7f40d2060700 1 -- 172.21.15.17:0/1994233701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc076380 con 0x7f40cc101dd0 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.660+0000 7f40d2060700 1 --2- 172.21.15.17:0/1994233701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40cc101dd0 0x7f40cc074490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40c8006010 tx=0x7f40c8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.660+0000 7f40c37fe700 1 -- 172.21.15.17:0/1994233701 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40c8026070 con 0x7f40cc101dd0 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.660+0000 7f40c37fe700 1 -- 172.21.15.17:0/1994233701 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40c800bc30 con 0x7f40cc101dd0 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.660+0000 7f40c37fe700 1 -- 172.21.15.17:0/1994233701 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40c80039d0 con 0x7f40cc101dd0 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.660+0000 7f40d42c4700 1 -- 172.21.15.17:0/1994233701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40cc074a30 con 0x7f40cc101dd0 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.660+0000 7f40d42c4700 1 -- 172.21.15.17:0/1994233701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40cc074ef0 con 0x7f40cc101dd0 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.661+0000 7f40c37fe700 1 -- 172.21.15.17:0/1994233701 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f40c8029070 con 0x7f40cc101dd0 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.661+0000 7f40d42c4700 1 -- 172.21.15.17:0/1994233701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40cc1032b0 con 0x7f40cc101dd0 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.661+0000 7f40c37fe700 1 --2- 172.21.15.17:0/1994233701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f40b8038fc0 0x7f40b803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:07.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.661+0000 7f40c37fe700 1 -- 172.21.15.17:0/1994233701 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f40c801e070 con 0x7f40cc101dd0 2024-03-24T02:52:07.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.661+0000 7f40d185f700 1 --2- 172.21.15.17:0/1994233701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f40b8038fc0 0x7f40b803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:07.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.661+0000 7f40d185f700 1 --2- 172.21.15.17:0/1994233701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f40b8038fc0 0x7f40b803b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f40bc006fd0 tx=0x7f40bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:07.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.666+0000 7f40c37fe700 1 -- 172.21.15.17:0/1994233701 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f40c8013070 con 0x7f40cc101dd0 2024-03-24T02:52:07.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.876+0000 7f40d42c4700 1 -- 172.21.15.17:0/1994233701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f40cc0759c0 con 0x7f40cc101dd0 2024-03-24T02:52:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.927+0000 7f40c37fe700 1 -- 172.21.15.17:0/1994233701 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f40c802bd20 con 0x7f40cc101dd0 2024-03-24T02:52:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.929+0000 7f40d42c4700 1 -- 172.21.15.17:0/1994233701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f40b8038fc0 msgr2=0x7f40b803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.929+0000 7f40d42c4700 1 --2- 172.21.15.17:0/1994233701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f40b8038fc0 0x7f40b803b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f40bc006fd0 tx=0x7f40bc006e40 comp rx=0 tx=0).stop 2024-03-24T02:52:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.929+0000 7f40d42c4700 1 -- 172.21.15.17:0/1994233701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40cc101dd0 msgr2=0x7f40cc074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.929+0000 7f40d42c4700 1 --2- 172.21.15.17:0/1994233701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40cc101dd0 0x7f40cc074490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40c8006010 tx=0x7f40c8005030 comp rx=0 tx=0).stop 2024-03-24T02:52:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.929+0000 7f40d42c4700 1 -- 172.21.15.17:0/1994233701 shutdown_connections 2024-03-24T02:52:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.929+0000 7f40d42c4700 1 --2- 172.21.15.17:0/1994233701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f40b8038fc0 0x7f40b803b480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.929+0000 7f40d42c4700 1 --2- 172.21.15.17:0/1994233701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40cc101dd0 0x7f40cc074490 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.929+0000 7f40d42c4700 1 -- 172.21.15.17:0/1994233701 >> 172.21.15.17:0/1994233701 conn(0x7f40cc0fd680 msgr2=0x7f40cc1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.929+0000 7f40d42c4700 1 -- 172.21.15.17:0/1994233701 shutdown_connections 2024-03-24T02:52:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:07.929+0000 7f40d42c4700 1 -- 172.21.15.17:0/1994233701 wait complete. 2024-03-24T02:52:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:adjusted tunables profile to default 2024-03-24T02:52:08.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:07 smithi017 ceph-mon[31926]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:08.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:07 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1994233701' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-24T02:52:08.641 INFO:tasks.cephadm:Adding mon.a on smithi017 2024-03-24T02:52:08.641 INFO:tasks.cephadm:Adding mon.b on smithi019 2024-03-24T02:52:08.641 INFO:tasks.cephadm:Adding mon.c on smithi188 2024-03-24T02:52:08.641 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi017:172.21.15.17=a;smithi019:172.21.15.19=b;smithi188:172.21.15.188=c' 2024-03-24T02:52:08.875 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:09.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:08 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1994233701' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-24T02:52:09.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:08 smithi017 ceph-mon[31926]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T02:52:09.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:09.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:09.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:09.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:09.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:52:09.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:09.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:09.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:08 smithi017 ceph-mon[31926]: Updating smithi188:/etc/ceph/ceph.conf 2024-03-24T02:52:09.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:08 smithi017 ceph-mon[31926]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:09.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:08 smithi017 ceph-mon[31926]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-03-24T02:52:10.365 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.365+0000 7fbf38b9a700 1 -- 172.21.15.188:0/2407313740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34101dd0 msgr2=0x7fbf341021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:10.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.365+0000 7fbf38b9a700 1 --2- 172.21.15.188:0/2407313740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34101dd0 0x7fbf341021b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400b0a0 tx=0x7fbf2400b3b0 comp rx=0 tx=0).stop 2024-03-24T02:52:10.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.365+0000 7fbf38b9a700 1 -- 172.21.15.188:0/2407313740 shutdown_connections 2024-03-24T02:52:10.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.366+0000 7fbf38b9a700 1 --2- 172.21.15.188:0/2407313740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34101dd0 0x7fbf341021b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:10.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.366+0000 7fbf38b9a700 1 -- 172.21.15.188:0/2407313740 >> 172.21.15.188:0/2407313740 conn(0x7fbf340fd6a0 msgr2=0x7fbf340ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:10.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.366+0000 7fbf38b9a700 1 -- 172.21.15.188:0/2407313740 shutdown_connections 2024-03-24T02:52:10.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.366+0000 7fbf38b9a700 1 -- 172.21.15.188:0/2407313740 wait complete. 2024-03-24T02:52:10.367 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.366+0000 7fbf38b9a700 1 Processor -- start 2024-03-24T02:52:10.367 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.366+0000 7fbf38b9a700 1 -- start start 2024-03-24T02:52:10.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.366+0000 7fbf38b9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34101dd0 0x7fbf3419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:10.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.366+0000 7fbf38b9a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3410c1a0 con 0x7fbf34101dd0 2024-03-24T02:52:10.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.367+0000 7fbf3259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34101dd0 0x7fbf3419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:10.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.367+0000 7fbf3259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34101dd0 0x7fbf3419f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.188:38698/0 (socket says 172.21.15.188:38698) 2024-03-24T02:52:10.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.367+0000 7fbf3259c700 1 -- 172.21.15.188:0/3347402891 learned_addr learned my addr 172.21.15.188:0/3347402891 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-24T02:52:10.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.367+0000 7fbf3259c700 1 -- 172.21.15.188:0/3347402891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3419fb00 con 0x7fbf34101dd0 2024-03-24T02:52:10.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.367+0000 7fbf3259c700 1 --2- 172.21.15.188:0/3347402891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34101dd0 0x7fbf3419f5c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf24015040 tx=0x7fbf24003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:10.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.368+0000 7fbf237fe700 1 -- 172.21.15.188:0/3347402891 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf24017040 con 0x7fbf34101dd0 2024-03-24T02:52:10.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.368+0000 7fbf38b9a700 1 -- 172.21.15.188:0/3347402891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3419fd90 con 0x7fbf34101dd0 2024-03-24T02:52:10.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.368+0000 7fbf237fe700 1 -- 172.21.15.188:0/3347402891 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf24022070 con 0x7fbf34101dd0 2024-03-24T02:52:10.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.368+0000 7fbf237fe700 1 -- 172.21.15.188:0/3347402891 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf2401d5d0 con 0x7fbf34101dd0 2024-03-24T02:52:10.368 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.368+0000 7fbf38b9a700 1 -- 172.21.15.188:0/3347402891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf34199690 con 0x7fbf34101dd0 2024-03-24T02:52:10.369 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.368+0000 7fbf38b9a700 1 -- 172.21.15.188:0/3347402891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf341032b0 con 0x7fbf34101dd0 2024-03-24T02:52:10.369 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.369+0000 7fbf237fe700 1 -- 172.21.15.188:0/3347402891 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fbf24004000 con 0x7fbf34101dd0 2024-03-24T02:52:10.369 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.369+0000 7fbf237fe700 1 --2- 172.21.15.188:0/3347402891 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbf1c038fc0 0x7fbf1c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:10.370 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.369+0000 7fbf237fe700 1 -- 172.21.15.188:0/3347402891 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbf2400f050 con 0x7fbf34101dd0 2024-03-24T02:52:10.370 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.369+0000 7fbf31d9b700 1 --2- 172.21.15.188:0/3347402891 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbf1c038fc0 0x7fbf1c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:10.370 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.369+0000 7fbf31d9b700 1 --2- 172.21.15.188:0/3347402891 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbf1c038fc0 0x7fbf1c03b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbf28006fd0 tx=0x7fbf28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:10.374 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.374+0000 7fbf237fe700 1 -- 172.21.15.188:0/3347402891 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbf24007e20 con 0x7fbf34101dd0 2024-03-24T02:52:10.580 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.580+0000 7fbf38b9a700 1 -- 172.21.15.188:0/3347402891 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi017:172.21.15.17=a;smithi019:172.21.15.19=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf3419abc0 con 0x7fbf1c038fc0 2024-03-24T02:52:10.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.586+0000 7fbf237fe700 1 -- 172.21.15.188:0/3347402891 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbf3419abc0 con 0x7fbf1c038fc0 2024-03-24T02:52:10.586 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mon update... 2024-03-24T02:52:10.588 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.588+0000 7fbf38b9a700 1 -- 172.21.15.188:0/3347402891 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbf1c038fc0 msgr2=0x7fbf1c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:10.588 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.588+0000 7fbf38b9a700 1 --2- 172.21.15.188:0/3347402891 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbf1c038fc0 0x7fbf1c03b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbf28006fd0 tx=0x7fbf28006e40 comp rx=0 tx=0).stop 2024-03-24T02:52:10.588 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.588+0000 7fbf38b9a700 1 -- 172.21.15.188:0/3347402891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34101dd0 msgr2=0x7fbf3419f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:10.588 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.588+0000 7fbf38b9a700 1 --2- 172.21.15.188:0/3347402891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34101dd0 0x7fbf3419f5c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf24015040 tx=0x7fbf24003d50 comp rx=0 tx=0).stop 2024-03-24T02:52:10.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.588+0000 7fbf38b9a700 1 -- 172.21.15.188:0/3347402891 shutdown_connections 2024-03-24T02:52:10.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.589+0000 7fbf38b9a700 1 --2- 172.21.15.188:0/3347402891 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbf1c038fc0 0x7fbf1c03b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:10.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.589+0000 7fbf38b9a700 1 --2- 172.21.15.188:0/3347402891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34101dd0 0x7fbf3419f5c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:10.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.589+0000 7fbf38b9a700 1 -- 172.21.15.188:0/3347402891 >> 172.21.15.188:0/3347402891 conn(0x7fbf340fd6a0 msgr2=0x7fbf340743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:10.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.589+0000 7fbf38b9a700 1 -- 172.21.15.188:0/3347402891 shutdown_connections 2024-03-24T02:52:10.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:10.589+0000 7fbf38b9a700 1 -- 172.21.15.188:0/3347402891 wait complete. 2024-03-24T02:52:10.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:10 smithi017 ceph-mon[31926]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T02:52:10.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:10 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:10.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:10 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:10.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:10 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:10.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:10 smithi017 ceph-mon[31926]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:11.190 DEBUG:teuthology.orchestra.run.smithi019:mon.b> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.b.service 2024-03-24T02:52:11.193 DEBUG:teuthology.orchestra.run.smithi188:mon.c> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.c.service 2024-03-24T02:52:11.195 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-24T02:52:11.195 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T02:52:11.852 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:11 smithi017 ceph-mon[31926]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi017:172.21.15.17=a;smithi019:172.21.15.19=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:52:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:11 smithi017 ceph-mon[31926]: Saving service mon spec with placement smithi017:172.21.15.17=a;smithi019:172.21.15.19=b;smithi188:172.21.15.188=c;count:3 2024-03-24T02:52:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:11 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:11 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:11 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:11 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:11 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:11 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:11 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:11 smithi017 ceph-mon[31926]: Deploying daemon mon.c on smithi188 2024-03-24T02:52:12.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:12 smithi017 ceph-mon[31926]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:13 smithi017 ceph-mon[31926]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:14.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.517+0000 7f21b7aab700 1 -- 172.21.15.188:0/1608104107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21b0106a60 msgr2=0x7f21b0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:14.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.517+0000 7f21b7aab700 1 --2- 172.21.15.188:0/1608104107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21b0106a60 0x7f21b0106e40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009910 tx=0x7f21a0009c20 comp rx=0 tx=0).stop 2024-03-24T02:52:14.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.517+0000 7f21b7aab700 1 -- 172.21.15.188:0/1608104107 shutdown_connections 2024-03-24T02:52:14.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.517+0000 7f21b7aab700 1 --2- 172.21.15.188:0/1608104107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21b0106a60 0x7f21b0106e40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:14.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.517+0000 7f21b7aab700 1 -- 172.21.15.188:0/1608104107 >> 172.21.15.188:0/1608104107 conn(0x7f21b00fc550 msgr2=0x7f21b00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:14.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.518+0000 7f21b7aab700 1 -- 172.21.15.188:0/1608104107 shutdown_connections 2024-03-24T02:52:14.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.518+0000 7f21b7aab700 1 -- 172.21.15.188:0/1608104107 wait complete. 2024-03-24T02:52:14.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.518+0000 7f21b7aab700 1 Processor -- start 2024-03-24T02:52:14.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.518+0000 7f21b7aab700 1 -- start start 2024-03-24T02:52:14.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.518+0000 7f21b7aab700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21b0106a60 0x7f21b0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:14.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.518+0000 7f21b7aab700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b010b0a0 con 0x7f21b0106a60 2024-03-24T02:52:14.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.519+0000 7f21b5847700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21b0106a60 0x7f21b0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:14.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.519+0000 7f21b5847700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21b0106a60 0x7f21b0198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.188:54788/0 (socket says 172.21.15.188:54788) 2024-03-24T02:52:14.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.519+0000 7f21b5847700 1 -- 172.21.15.188:0/675204674 learned_addr learned my addr 172.21.15.188:0/675204674 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-24T02:52:14.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.519+0000 7f21b5847700 1 -- 172.21.15.188:0/675204674 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b01989b0 con 0x7f21b0106a60 2024-03-24T02:52:14.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.519+0000 7f21b5847700 1 --2- 172.21.15.188:0/675204674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21b0106a60 0x7f21b0198470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f21a0005850 tx=0x7f21a0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:14.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.519+0000 7f21a6ffd700 1 -- 172.21.15.188:0/675204674 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21a0026070 con 0x7f21b0106a60 2024-03-24T02:52:14.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.520+0000 7f21b7aab700 1 -- 172.21.15.188:0/675204674 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b0198ca0 con 0x7f21b0106a60 2024-03-24T02:52:14.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.520+0000 7f21b7aab700 1 -- 172.21.15.188:0/675204674 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b0199160 con 0x7f21b0106a60 2024-03-24T02:52:14.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.520+0000 7f21a6ffd700 1 -- 172.21.15.188:0/675204674 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21a000bbd0 con 0x7f21b0106a60 2024-03-24T02:52:14.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.520+0000 7f21a6ffd700 1 -- 172.21.15.188:0/675204674 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21a00039b0 con 0x7f21b0106a60 2024-03-24T02:52:14.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.520+0000 7f21b7aab700 1 -- 172.21.15.188:0/675204674 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2194005360 con 0x7f21b0106a60 2024-03-24T02:52:14.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.521+0000 7f21a6ffd700 1 -- 172.21.15.188:0/675204674 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f21a0029070 con 0x7f21b0106a60 2024-03-24T02:52:14.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.521+0000 7f21a6ffd700 1 --2- 172.21.15.188:0/675204674 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f219c038fc0 0x7f219c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:14.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.521+0000 7f21a6ffd700 1 -- 172.21.15.188:0/675204674 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f21a001e070 con 0x7f21b0106a60 2024-03-24T02:52:14.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.521+0000 7f21b5046700 1 --2- 172.21.15.188:0/675204674 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f219c038fc0 0x7f219c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:14.525 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.522+0000 7f21b5046700 1 --2- 172.21.15.188:0/675204674 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f219c038fc0 0x7f219c03b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f21ac006fd0 tx=0x7f21ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:14.526 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.526+0000 7f21a6ffd700 1 -- 172.21.15.188:0/675204674 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f21a0013070 con 0x7f21b0106a60 2024-03-24T02:52:14.826 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.825+0000 7f21b7aab700 1 -- 172.21.15.188:0/675204674 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2194005650 con 0x7f21b0106a60 2024-03-24T02:52:14.826 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.826+0000 7f21a6ffd700 1 -- 172.21.15.188:0/675204674 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f21a0004390 con 0x7f21b0106a60 2024-03-24T02:52:14.826 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:52:14.827 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","modified":"2024-03-24T02:50:32.086168Z","created":"2024-03-24T02:50:32.086168Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T02:52:14.830 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.828+0000 7f21b7aab700 1 -- 172.21.15.188:0/675204674 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f219c038fc0 msgr2=0x7f219c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:14.830 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.828+0000 7f21b7aab700 1 --2- 172.21.15.188:0/675204674 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f219c038fc0 0x7f219c03b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f21ac006fd0 tx=0x7f21ac006e40 comp rx=0 tx=0).stop 2024-03-24T02:52:14.830 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.828+0000 7f21b7aab700 1 -- 172.21.15.188:0/675204674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21b0106a60 msgr2=0x7f21b0198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:14.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.828+0000 7f21b7aab700 1 --2- 172.21.15.188:0/675204674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21b0106a60 0x7f21b0198470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f21a0005850 tx=0x7f21a0005030 comp rx=0 tx=0).stop 2024-03-24T02:52:14.835 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.833+0000 7f21b7aab700 1 -- 172.21.15.188:0/675204674 shutdown_connections 2024-03-24T02:52:14.835 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.833+0000 7f21b7aab700 1 --2- 172.21.15.188:0/675204674 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f219c038fc0 0x7f219c03b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:14.835 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.833+0000 7f21b7aab700 1 --2- 172.21.15.188:0/675204674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21b0106a60 0x7f21b0198470 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:14.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.833+0000 7f21b7aab700 1 -- 172.21.15.188:0/675204674 >> 172.21.15.188:0/675204674 conn(0x7f21b00fc550 msgr2=0x7f21b0108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:14.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.837+0000 7f21b7aab700 1 -- 172.21.15.188:0/675204674 shutdown_connections 2024-03-24T02:52:14.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:14.837+0000 7f21b7aab700 1 -- 172.21.15.188:0/675204674 wait complete. 2024-03-24T02:52:14.837 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2024-03-24T02:52:15.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:14 smithi188 systemd[1]: Starting Ceph mon.c for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:52:15.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:14 smithi017 ceph-mon[31926]: from='client.? 172.21.15.188:0/675204674' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T02:52:15.755 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 podman[36187]: 2024-03-24 02:52:15.569922964 +0000 UTC m=+0.101310422 container create 6e9e3f6184421e1100758624dfcd714c15c89e0b98fe408b2a47b830922bf2d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-24T02:52:15.755 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 podman[36187]: 2024-03-24 02:52:15.486226059 +0000 UTC m=+0.017613513 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T02:52:15.755 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 podman[36187]: 2024-03-24 02:52:15.755508198 +0000 UTC m=+0.286895646 container init 6e9e3f6184421e1100758624dfcd714c15c89e0b98fe408b2a47b830922bf2d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-c, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-24T02:52:16.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 podman[36187]: 2024-03-24 02:52:15.758545666 +0000 UTC m=+0.289933096 container start 6e9e3f6184421e1100758624dfcd714c15c89e0b98fe408b2a47b830922bf2d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-c, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-24T02:52:16.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T02:52:16.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-24T02:52:16.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: pidfile_write: ignore empty --pid-file 2024-03-24T02:52:16.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: load: jerasure load: lrc 2024-03-24T02:52:16.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: RocksDB version: 7.9.2 2024-03-24T02:52:16.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Git sha 0 2024-03-24T02:52:16.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-24T02:52:16.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: DB SUMMARY 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: DB Session ID: MN64U0JAENK6K9PAXGTT 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: CURRENT file: CURRENT 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.error_if_exists: 0 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.create_if_missing: 0 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.paranoid_checks: 1 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.env: 0x55f9f9648b20 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T02:52:16.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.info_log: 0x55f9fb981920 2024-03-24T02:52:16.050 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T02:52:16.050 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.statistics: (nil) 2024-03-24T02:52:16.050 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.use_fsync: 0 2024-03-24T02:52:16.050 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_log_file_size: 0 2024-03-24T02:52:16.050 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T02:52:16.050 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T02:52:16.050 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T02:52:16.050 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T02:52:16.050 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.allow_fallocate: 1 2024-03-24T02:52:16.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T02:52:16.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T02:52:16.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.use_direct_reads: 0 2024-03-24T02:52:16.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T02:52:16.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T02:52:16.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.db_log_dir: 2024-03-24T02:52:16.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.wal_dir: 2024-03-24T02:52:16.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T02:52:16.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T02:52:16.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T02:52:16.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T02:52:16.052 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.write_buffer_manager: 0x55f9fb9541e0 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.unordered_write: 0 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T02:52:16.053 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T02:52:16.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T02:52:16.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T02:52:16.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.row_cache: None 2024-03-24T02:52:16.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.wal_filter: None 2024-03-24T02:52:16.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T02:52:16.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T02:52:16.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.two_write_queues: 0 2024-03-24T02:52:16.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T02:52:16.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.wal_compression: 0 2024-03-24T02:52:16.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.atomic_flush: 0 2024-03-24T02:52:16.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T02:52:16.054 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.log_readahead_size: 0 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_background_jobs: 2 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_background_compactions: -1 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_subcompactions: 1 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T02:52:16.055 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_open_files: -1 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_background_flushes: -1 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Compression algorithms supported: 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: kZSTD supported: 0 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: kXpressCompression supported: 0 2024-03-24T02:52:16.056 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: kZlibCompression supported: 1 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: kSnappyCompression supported: 1 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: kLZ4Compression supported: 1 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: kBZip2Compression supported: 0 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.merge_operator: 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_filter: None 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T02:52:16.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f9fb981a20) 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: cache_index_and_filter_blocks: 1 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: pin_top_level_index_and_filter: 1 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: index_type: 0 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: data_block_index_type: 0 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: index_shortening: 1 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: checksum: 4 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: no_block_cache: 0 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache: 0x55f9fb987090 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache_name: BinnedLRUCache 2024-03-24T02:52:16.058 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache_options: 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: capacity : 536870912 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: num_shard_bits : 4 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: strict_capacity_limit : 0 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: high_pri_pool_ratio: 0.000 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache_compressed: (nil) 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: persistent_cache: (nil) 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_size: 4096 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_size_deviation: 10 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_restart_interval: 16 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: index_block_restart_interval: 1 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: metadata_block_size: 4096 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: partition_filters: 0 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: use_delta_encoding: 1 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: filter_policy: bloomfilter 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: whole_key_filtering: 1 2024-03-24T02:52:16.059 INFO:journalctl@ceph.mon.c.smithi188.stdout: verify_compression: 0 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout: read_amp_bytes_per_bit: 0 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout: format_version: 5 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout: enable_index_compression: 1 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_align: 0 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout: max_auto_readahead_size: 262144 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout: prepopulate_block_cache: 0 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout: initial_auto_readahead_size: 8192 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compression: NoCompression 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T02:52:16.060 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.num_levels: 7 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T02:52:16.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T02:52:16.062 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T02:52:16.063 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.table_properties_collectors: 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.inplace_update_support: 0 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T02:52:16.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.bloom_locality: 0 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.max_successive_merges: 0 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.ttl: 2592000 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.enable_blob_files: false 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.min_blob_size: 0 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T02:52:16.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6b65275d-793a-4dde-94de-0ca5e1fe799f 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711248735787629, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711248735788072, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711248735, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6b65275d-793a-4dde-94de-0ca5e1fe799f", "db_session_id": "MN64U0JAENK6K9PAXGTT", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711248735788152, "job": 1, "event": "recovery_finished"} 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T02:52:16.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f9fba7a000 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: DB pointer 0x55f9fba64000 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** DB Stats ** 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** Compaction Stats [default] ** 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T02:52:16.067 INFO:journalctl@ceph.mon.c.smithi188.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** Compaction Stats [default] ** 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T02:52:16.068 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout: Block cache BinnedLRUCache@0x55f9fb987090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 7e-06 secs_since: 0 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: using public_addr v2:172.21.15.188:0/0 -> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: starting mon.c rank -1 at public addrs [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] at bind addrs [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(???) e0 preinit fsid 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).mds e1 new map 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-24T02:52:16.069 INFO:journalctl@ceph.mon.c.smithi188.stdout: e1 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout: legacy client fscid: -1 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout: No filesystems configured 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T02:52:16.070 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:16.071 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: Deploying cephadm binary to smithi188 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: Added host smithi188 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1994233701' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1994233701' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: Updating smithi188:/etc/ceph/ceph.conf 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-03-24T02:52:16.073 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi017:172.21.15.17=a;smithi019:172.21.15.19=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: Saving service mon spec with placement smithi017:172.21.15.17=a;smithi019:172.21.15.19=b;smithi188:172.21.15.188=c;count:3 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:16.074 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: Deploying daemon mon.c on smithi188 2024-03-24T02:52:16.075 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:16.075 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:16.075 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: from='client.? 172.21.15.188:0/675204674' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T02:52:16.075 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 ceph-mon[36206]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-24T02:52:16.075 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 bash[36187]: 6e9e3f6184421e1100758624dfcd714c15c89e0b98fe408b2a47b830922bf2d4 2024-03-24T02:52:16.075 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:15 smithi188 systemd[1]: Started Ceph mon.c for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:52:16.469 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-24T02:52:16.469 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T02:52:16.703 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T02:52:21.115 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.115+0000 7f699e72a700 1 -- 172.21.15.188:0/4270525269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69800035c0 msgr2=0x7f6980005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:21.115 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.115+0000 7f699e72a700 1 --2- 172.21.15.188:0/4270525269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69800035c0 0x7f6980005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6998101270 tx=0x7f698800feb0 comp rx=0 tx=0).stop 2024-03-24T02:52:21.115 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.115+0000 7f699e72a700 1 -- 172.21.15.188:0/4270525269 shutdown_connections 2024-03-24T02:52:21.115 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.115+0000 7f699e72a700 1 --2- 172.21.15.188:0/4270525269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69800035c0 0x7f6980005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:21.115 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.115+0000 7f699e72a700 1 -- 172.21.15.188:0/4270525269 >> 172.21.15.188:0/4270525269 conn(0x7f6998077280 msgr2=0x7f6998077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:21.115 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.115+0000 7f699e72a700 1 -- 172.21.15.188:0/4270525269 shutdown_connections 2024-03-24T02:52:21.115 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.115+0000 7f699e72a700 1 -- 172.21.15.188:0/4270525269 wait complete. 2024-03-24T02:52:21.115 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.116+0000 7f699e72a700 1 Processor -- start 2024-03-24T02:52:21.115 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.116+0000 7f699e72a700 1 -- start start 2024-03-24T02:52:21.116 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.116+0000 7f699e72a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69800035c0 0x7f69981013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.116+0000 7f699e72a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69981018f0 0x7f6998101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.116+0000 7f699e72a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6998109d00 con 0x7f69981018f0 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.116+0000 7f699e72a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6998109b80 con 0x7f69800035c0 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.116+0000 7f6997fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69800035c0 0x7f69981013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.116+0000 7f6997fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69800035c0 0x7f69981013b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:37950/0 (socket says 172.21.15.188:37950) 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.116+0000 7f6997fff700 1 -- 172.21.15.188:0/3201245534 learned_addr learned my addr 172.21.15.188:0/3201245534 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.117+0000 7f6997fff700 1 -- 172.21.15.188:0/3201245534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69800035c0 msgr2=0x7f69981013b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.117+0000 7f6997fff700 1 -- 172.21.15.188:0/3201245534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69800035c0 msgr2=0x7f69981013b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.117+0000 7f6997fff700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69800035c0 0x7f69981013b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.117+0000 7f6997fff700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69800035c0 0x7f69981013b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.117+0000 7f69977fe700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69981018f0 0x7f6998101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.117+0000 7f69977fe700 1 -- 172.21.15.188:0/3201245534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69800035c0 msgr2=0x7f69981013b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.117+0000 7f69977fe700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69800035c0 0x7f69981013b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:21.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.117+0000 7f69977fe700 1 -- 172.21.15.188:0/3201245534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69981032d0 con 0x7f69981018f0 2024-03-24T02:52:21.118 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.117+0000 7f69977fe700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69981018f0 0x7f6998101d70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f698c00b4d0 tx=0x7f698c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:21.118 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.117+0000 7f69957fa700 1 -- 172.21.15.188:0/3201245534 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f698c011630 con 0x7f69981018f0 2024-03-24T02:52:21.118 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.118+0000 7f69957fa700 1 -- 172.21.15.188:0/3201245534 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f698c011cb0 con 0x7f69981018f0 2024-03-24T02:52:21.118 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.118+0000 7f69957fa700 1 -- 172.21.15.188:0/3201245534 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f698c010e60 con 0x7f69981018f0 2024-03-24T02:52:21.118 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.118+0000 7f699e72a700 1 -- 172.21.15.188:0/3201245534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69981a3b10 con 0x7f69981018f0 2024-03-24T02:52:21.118 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.118+0000 7f699e72a700 1 -- 172.21.15.188:0/3201245534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69981a4050 con 0x7f69981018f0 2024-03-24T02:52:21.119 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.119+0000 7f69957fa700 1 -- 172.21.15.188:0/3201245534 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f698c018020 con 0x7f69981018f0 2024-03-24T02:52:21.119 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.119+0000 7f699e72a700 1 -- 172.21.15.188:0/3201245534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69981034f0 con 0x7f69981018f0 2024-03-24T02:52:21.119 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.119+0000 7f69957fa700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6984038d60 0x7f698403b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:21.119 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.119+0000 7f69957fa700 1 -- 172.21.15.188:0/3201245534 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f698c04b970 con 0x7f69981018f0 2024-03-24T02:52:21.119 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.119+0000 7f6997fff700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6984038d60 0x7f698403b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:21.119 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.120+0000 7f6997fff700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6984038d60 0x7f698403b220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f698800fa60 tx=0x7f6988005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:21.123 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.125+0000 7f69957fa700 1 -- 172.21.15.188:0/3201245534 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f698c01f020 con 0x7f69981018f0 2024-03-24T02:52:21.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:52:21.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: mon.a calling monitor election 2024-03-24T02:52:21.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:21.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: mon.c calling monitor election 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: monmap e2: 2 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: fsmap 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: mgrmap e13: x(active, since 50s) 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: overall HEALTH_OK 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:21.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:21.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:21.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:21.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:52:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: mon.a calling monitor election 2024-03-24T02:52:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: mon.c calling monitor election 2024-03-24T02:52:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:21.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-24T02:52:21.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: monmap e2: 2 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T02:52:21.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: fsmap 2024-03-24T02:52:21.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T02:52:21.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: mgrmap e13: x(active, since 50s) 2024-03-24T02:52:21.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: overall HEALTH_OK 2024-03-24T02:52:21.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:21.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:21.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:21.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:21.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:21.400 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.401+0000 7f699e72a700 1 -- 172.21.15.188:0/3201245534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f69981a43d0 con 0x7f69981018f0 2024-03-24T02:52:21.401 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.402+0000 7f69957fa700 1 -- 172.21.15.188:0/3201245534 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f698c018310 con 0x7f69981018f0 2024-03-24T02:52:21.401 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:52:21.402 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":2,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","modified":"2024-03-24T02:52:15.810772Z","created":"2024-03-24T02:50:32.086168Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-24T02:52:21.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.404+0000 7f699e72a700 1 -- 172.21.15.188:0/3201245534 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6984038d60 msgr2=0x7f698403b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:21.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.404+0000 7f699e72a700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6984038d60 0x7f698403b220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f698800fa60 tx=0x7f6988005a90 comp rx=0 tx=0).stop 2024-03-24T02:52:21.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.404+0000 7f699e72a700 1 -- 172.21.15.188:0/3201245534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69981018f0 msgr2=0x7f6998101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:21.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.404+0000 7f699e72a700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69981018f0 0x7f6998101d70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f698c00b4d0 tx=0x7f698c00b890 comp rx=0 tx=0).stop 2024-03-24T02:52:21.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.404+0000 7f699e72a700 1 -- 172.21.15.188:0/3201245534 shutdown_connections 2024-03-24T02:52:21.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.404+0000 7f699e72a700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6984038d60 0x7f698403b220 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:21.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.404+0000 7f699e72a700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69800035c0 0x7f69981013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:21.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.404+0000 7f699e72a700 1 --2- 172.21.15.188:0/3201245534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69981018f0 0x7f6998101d70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:21.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.404+0000 7f699e72a700 1 -- 172.21.15.188:0/3201245534 >> 172.21.15.188:0/3201245534 conn(0x7f6998077280 msgr2=0x7f6998104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:21.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.405+0000 7f699e72a700 1 -- 172.21.15.188:0/3201245534 shutdown_connections 2024-03-24T02:52:21.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:21.405+0000 7f699e72a700 1 -- 172.21.15.188:0/3201245534 wait complete. 2024-03-24T02:52:21.404 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 2 2024-03-24T02:52:21.999 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:21 smithi188 ceph-mon[36206]: Deploying daemon mon.b on smithi019 2024-03-24T02:52:21.999 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:21 smithi188 ceph-mon[36206]: from='client.? 172.21.15.188:0/3201245534' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T02:52:21.999 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:21 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:21.999 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:21 smithi188 ceph-mon[36206]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:21 smithi017 ceph-mon[31926]: Deploying daemon mon.b on smithi019 2024-03-24T02:52:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:21 smithi017 ceph-mon[31926]: from='client.? 172.21.15.188:0/3201245534' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T02:52:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:21 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:21 smithi017 ceph-mon[31926]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:23.025 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-24T02:52:23.026 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T02:52:23.206 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:52:22 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:52:22.810+0000 7f5f0f6a2700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-24T02:52:23.267 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T02:52:24.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:23 smithi188 ceph-mon[36206]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:24.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:23 smithi017 ceph-mon[31926]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:24.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.741+0000 7f4f7fb48700 1 -- 172.21.15.188:0/2726803326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f78100df0 msgr2=0x7f4f781011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:24.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.741+0000 7f4f7fb48700 1 --2- 172.21.15.188:0/2726803326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f78100df0 0x7f4f781011d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009910 tx=0x7f4f74009c20 comp rx=0 tx=0).stop 2024-03-24T02:52:24.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.741+0000 7f4f7fb48700 1 -- 172.21.15.188:0/2726803326 shutdown_connections 2024-03-24T02:52:24.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.741+0000 7f4f7fb48700 1 --2- 172.21.15.188:0/2726803326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f78100df0 0x7f4f781011d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:24.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.741+0000 7f4f7fb48700 1 -- 172.21.15.188:0/2726803326 >> 172.21.15.188:0/2726803326 conn(0x7f4f780fc6a0 msgr2=0x7f4f780feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:24.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.741+0000 7f4f7fb48700 1 -- 172.21.15.188:0/2726803326 shutdown_connections 2024-03-24T02:52:24.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.741+0000 7f4f7fb48700 1 -- 172.21.15.188:0/2726803326 wait complete. 2024-03-24T02:52:24.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.742+0000 7f4f7fb48700 1 Processor -- start 2024-03-24T02:52:24.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.742+0000 7f4f7fb48700 1 -- start start 2024-03-24T02:52:24.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.742+0000 7f4f7fb48700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78100df0 0x7f4f78102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:24.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.742+0000 7f4f7fb48700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f78102dd0 0x7f4f78103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:24.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.742+0000 7f4f7fb48700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7810b1c0 con 0x7f4f78102dd0 2024-03-24T02:52:24.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.742+0000 7f4f7fb48700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7810b040 con 0x7f4f78100df0 2024-03-24T02:52:24.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.742+0000 7f4f7d8e4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78100df0 0x7f4f78102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:24.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.742+0000 7f4f7d8e4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78100df0 0x7f4f78102890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:59466/0 (socket says 172.21.15.188:59466) 2024-03-24T02:52:24.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.742+0000 7f4f7d8e4700 1 -- 172.21.15.188:0/3039722249 learned_addr learned my addr 172.21.15.188:0/3039722249 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-24T02:52:24.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.742+0000 7f4f7d0e3700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f78102dd0 0x7f4f78103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:24.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.742+0000 7f4f7d8e4700 1 -- 172.21.15.188:0/3039722249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78100df0 msgr2=0x7f4f78102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-24T02:52:24.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.742+0000 7f4f7d8e4700 1 -- 172.21.15.188:0/3039722249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78100df0 msgr2=0x7f4f78102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-24T02:52:24.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.743+0000 7f4f7d8e4700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78100df0 0x7f4f78102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-24T02:52:24.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.743+0000 7f4f7d8e4700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78100df0 0x7f4f78102890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T02:52:24.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.743+0000 7f4f7d0e3700 1 -- 172.21.15.188:0/3039722249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78100df0 msgr2=0x7f4f78102890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T02:52:24.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.743+0000 7f4f7d0e3700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78100df0 0x7f4f78102890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:24.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.743+0000 7f4f7d0e3700 1 -- 172.21.15.188:0/3039722249 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f781047b0 con 0x7f4f78102dd0 2024-03-24T02:52:24.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.743+0000 7f4f7d0e3700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f78102dd0 0x7f4f78103250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800b4d0 tx=0x7f4f6800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:24.746 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.743+0000 7f4f6effd700 1 -- 172.21.15.188:0/3039722249 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4f68011630 con 0x7f4f78102dd0 2024-03-24T02:52:24.746 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.743+0000 7f4f6effd700 1 -- 172.21.15.188:0/3039722249 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f68011cb0 con 0x7f4f78102dd0 2024-03-24T02:52:24.746 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.743+0000 7f4f7fb48700 1 -- 172.21.15.188:0/3039722249 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f781a3c10 con 0x7f4f78102dd0 2024-03-24T02:52:24.746 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.743+0000 7f4f6effd700 1 -- 172.21.15.188:0/3039722249 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4f68010e60 con 0x7f4f78102dd0 2024-03-24T02:52:24.746 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.743+0000 7f4f7fb48700 1 -- 172.21.15.188:0/3039722249 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f781a4150 con 0x7f4f78102dd0 2024-03-24T02:52:24.746 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.744+0000 7f4f7fb48700 1 -- 172.21.15.188:0/3039722249 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f781022d0 con 0x7f4f78102dd0 2024-03-24T02:52:24.746 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.744+0000 7f4f6effd700 1 -- 172.21.15.188:0/3039722249 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f4f68018020 con 0x7f4f78102dd0 2024-03-24T02:52:24.746 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.744+0000 7f4f6effd700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4f64038d60 0x7f4f6403b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:24.746 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.744+0000 7f4f6effd700 1 -- 172.21.15.188:0/3039722249 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4f6804b9e0 con 0x7f4f78102dd0 2024-03-24T02:52:24.746 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.744+0000 7f4f7d8e4700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4f64038d60 0x7f4f6403b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:24.746 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.745+0000 7f4f7d8e4700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4f64038d60 0x7f4f6403b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400b5c0 tx=0x7f4f74009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:24.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.750+0000 7f4f6effd700 1 -- 172.21.15.188:0/3039722249 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4f6801f020 con 0x7f4f78102dd0 2024-03-24T02:52:24.774 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:24.774+0000 7f4f6effd700 1 -- 172.21.15.188:0/3039722249 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f680179b0 con 0x7f4f78102dd0 2024-03-24T02:52:25.025 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:25.025+0000 7f4f7fb48700 1 -- 172.21.15.188:0/3039722249 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4f781a4460 con 0x7f4f78102dd0 2024-03-24T02:52:25.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:24 smithi019 systemd[1]: Started Ceph mon.b for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:52:29.807 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:29.807+0000 7f4f6effd700 1 -- 172.21.15.188:0/3039722249 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1279 (secure 0 0 0) 0x7f4f68015c60 con 0x7f4f78102dd0 2024-03-24T02:52:29.807 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-24T02:52:29.807 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":3,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","modified":"2024-03-24T02:52:24.774812Z","created":"2024-03-24T02:50:32.086168Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-24T02:52:29.810 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:29.810+0000 7f4f7fb48700 1 -- 172.21.15.188:0/3039722249 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4f64038d60 msgr2=0x7f4f6403b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:29.810 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:29.810+0000 7f4f7fb48700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4f64038d60 0x7f4f6403b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400b5c0 tx=0x7f4f74009650 comp rx=0 tx=0).stop 2024-03-24T02:52:29.810 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:29.810+0000 7f4f7fb48700 1 -- 172.21.15.188:0/3039722249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f78102dd0 msgr2=0x7f4f78103250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:29.810 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:29.810+0000 7f4f7fb48700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f78102dd0 0x7f4f78103250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800b4d0 tx=0x7f4f6800b890 comp rx=0 tx=0).stop 2024-03-24T02:52:29.810 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:29.810+0000 7f4f7fb48700 1 -- 172.21.15.188:0/3039722249 shutdown_connections 2024-03-24T02:52:29.810 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:29.810+0000 7f4f7fb48700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4f64038d60 0x7f4f6403b220 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:29.810 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:29.810+0000 7f4f7fb48700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78100df0 0x7f4f78102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:29.810 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:29.810+0000 7f4f7fb48700 1 --2- 172.21.15.188:0/3039722249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f78102dd0 0x7f4f78103250 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:29.810 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:29.810+0000 7f4f7fb48700 1 -- 172.21.15.188:0/3039722249 >> 172.21.15.188:0/3039722249 conn(0x7f4f780fc6a0 msgr2=0x7f4f7806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:29.811 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:29.810+0000 7f4f7fb48700 1 -- 172.21.15.188:0/3039722249 shutdown_connections 2024-03-24T02:52:29.811 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:29.810+0000 7f4f7fb48700 1 -- 172.21.15.188:0/3039722249 wait complete. 2024-03-24T02:52:29.811 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 3 2024-03-24T02:52:30.144 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:52:30.144 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: mon.c calling monitor election 2024-03-24T02:52:30.144 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: mon.a calling monitor election 2024-03-24T02:52:30.144 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:30.144 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:30.144 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='client.? 172.21.15.188:0/3039722249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T02:52:30.144 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:30.144 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:30.144 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:30.144 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:30.144 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: fsmap 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: mgrmap e13: x(active, since 59s) 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: mon.b (rank 2) addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] is down (out of quorum) 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:30.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:30.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: mon.c calling monitor election 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: mon.a calling monitor election 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='client.? 172.21.15.188:0/3039722249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-24T02:52:30.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T02:52:30.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: fsmap 2024-03-24T02:52:30.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T02:52:30.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: mgrmap e13: x(active, since 59s) 2024-03-24T02:52:30.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-24T02:52:30.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-24T02:52:30.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-24T02:52:30.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: mon.b (rank 2) addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] is down (out of quorum) 2024-03-24T02:52:30.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:30.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:30.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:30.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:30.470 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-24T02:52:30.470 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-24T02:52:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:52:31.037 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:30 smithi188 ceph-mon[36206]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:31.038 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:31.038 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:31.038 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:31.107 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:30 smithi017 ceph-mon[31926]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:31.107 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:31.107 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:31.107 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:32.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:52:32.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: mon.c calling monitor election 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: mon.a calling monitor election 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='client.? 172.21.15.188:0/3039722249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-24T02:52:32.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: fsmap 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: mgrmap e13: x(active, since 59s) 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: mon.b (rank 2) addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] is down (out of quorum) 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:32.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.328+0000 7f6d1adab700 1 -- 172.21.15.17:0/2173205471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d14106a60 msgr2=0x7f6d14106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.328+0000 7f6d1adab700 1 --2- 172.21.15.17:0/2173205471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d14106a60 0x7f6d14106e40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009910 tx=0x7f6d08009c20 comp rx=0 tx=0).stop 2024-03-24T02:52:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.328+0000 7f6d1adab700 1 -- 172.21.15.17:0/2173205471 shutdown_connections 2024-03-24T02:52:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.328+0000 7f6d1adab700 1 --2- 172.21.15.17:0/2173205471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d14106a60 0x7f6d14106e40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.328+0000 7f6d1adab700 1 -- 172.21.15.17:0/2173205471 >> 172.21.15.17:0/2173205471 conn(0x7f6d140776d0 msgr2=0x7f6d140fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.328+0000 7f6d1adab700 1 -- 172.21.15.17:0/2173205471 shutdown_connections 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.328+0000 7f6d1adab700 1 -- 172.21.15.17:0/2173205471 wait complete. 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d1adab700 1 Processor -- start 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d1adab700 1 -- start start 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d1adab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d14106a60 0x7f6d1419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d1adab700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d1419dba0 0x7f6d14197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d1adab700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d14197c70 0x7f6d14198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d1adab700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1410b050 con 0x7f6d14197c70 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d1adab700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1410aed0 con 0x7f6d14106a60 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d1adab700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1410b1d0 con 0x7f6d1419dba0 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d19348700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d14197c70 0x7f6d14198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d19348700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d14197c70 0x7f6d14198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50910/0 (socket says 172.21.15.17:50910) 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d19348700 1 -- 172.21.15.17:0/1812793976 learned_addr learned my addr 172.21.15.17:0/1812793976 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d18b47700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d14106a60 0x7f6d1419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d13fff700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d1419dba0 0x7f6d14197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d19348700 1 -- 172.21.15.17:0/1812793976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d1419dba0 msgr2=0x7f6d14197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d19348700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d1419dba0 0x7f6d14197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d19348700 1 -- 172.21.15.17:0/1812793976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d14106a60 msgr2=0x7f6d1419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d19348700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d14106a60 0x7f6d1419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.329+0000 7f6d19348700 1 -- 172.21.15.17:0/1812793976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d141a4410 con 0x7f6d14197c70 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.330+0000 7f6d19348700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d14197c70 0x7f6d14198120 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00ba50 tx=0x7f6d0c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.330+0000 7f6d11ffb700 1 -- 172.21.15.17:0/1812793976 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c019070 con 0x7f6d14197c70 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.330+0000 7f6d1adab700 1 -- 172.21.15.17:0/1812793976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d141a46a0 con 0x7f6d14197c70 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.330+0000 7f6d11ffb700 1 -- 172.21.15.17:0/1812793976 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d0c014440 con 0x7f6d14197c70 2024-03-24T02:52:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.330+0000 7f6d11ffb700 1 -- 172.21.15.17:0/1812793976 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0075b0 con 0x7f6d14197c70 2024-03-24T02:52:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.330+0000 7f6d1adab700 1 -- 172.21.15.17:0/1812793976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d141a4b90 con 0x7f6d14197c70 2024-03-24T02:52:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.331+0000 7f6d11ffb700 1 -- 172.21.15.17:0/1812793976 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f6d0c01c070 con 0x7f6d14197c70 2024-03-24T02:52:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.331+0000 7f6d11ffb700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6d04039450 0x7f6d0403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.331+0000 7f6d11ffb700 1 -- 172.21.15.17:0/1812793976 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6d0c04b610 con 0x7f6d14197c70 2024-03-24T02:52:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.331+0000 7f6d1adab700 1 -- 172.21.15.17:0/1812793976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d141021a0 con 0x7f6d14197c70 2024-03-24T02:52:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.331+0000 7f6d18b47700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6d04039450 0x7f6d0403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.331+0000 7f6d18b47700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6d04039450 0x7f6d0403b910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009750 tx=0x7f6d0800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.336+0000 7f6d11ffb700 1 -- 172.21.15.17:0/1812793976 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6d0c012740 con 0x7f6d14197c70 2024-03-24T02:52:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.544+0000 7f6d1adab700 1 -- 172.21.15.17:0/1812793976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6d14198c60 con 0x7f6d14197c70 2024-03-24T02:52:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.544+0000 7f6d11ffb700 1 -- 172.21.15.17:0/1812793976 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f6d0c014980 con 0x7f6d14197c70 2024-03-24T02:52:32.544 INFO:teuthology.orchestra.run.smithi017.stdout:# minimal ceph.conf for 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:52:32.544 INFO:teuthology.orchestra.run.smithi017.stdout:[global] 2024-03-24T02:52:32.544 INFO:teuthology.orchestra.run.smithi017.stdout: fsid = 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T02:52:32.544 INFO:teuthology.orchestra.run.smithi017.stdout: mon_host = [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2024-03-24T02:52:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.546+0000 7f6d1adab700 1 -- 172.21.15.17:0/1812793976 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6d04039450 msgr2=0x7f6d0403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.546+0000 7f6d1adab700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6d04039450 0x7f6d0403b910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009750 tx=0x7f6d0800b540 comp rx=0 tx=0).stop 2024-03-24T02:52:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.546+0000 7f6d1adab700 1 -- 172.21.15.17:0/1812793976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d14197c70 msgr2=0x7f6d14198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.546+0000 7f6d1adab700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d14197c70 0x7f6d14198120 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00ba50 tx=0x7f6d0c00bd60 comp rx=0 tx=0).stop 2024-03-24T02:52:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.546+0000 7f6d1adab700 1 -- 172.21.15.17:0/1812793976 shutdown_connections 2024-03-24T02:52:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.546+0000 7f6d1adab700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6d04039450 0x7f6d0403b910 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.546+0000 7f6d1adab700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d14197c70 0x7f6d14198120 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.546+0000 7f6d1adab700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d14106a60 0x7f6d1419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:32.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.546+0000 7f6d1adab700 1 --2- 172.21.15.17:0/1812793976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d1419dba0 0x7f6d14197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:32.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.546+0000 7f6d1adab700 1 -- 172.21.15.17:0/1812793976 >> 172.21.15.17:0/1812793976 conn(0x7f6d140776d0 msgr2=0x7f6d14104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:32.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.546+0000 7f6d1adab700 1 -- 172.21.15.17:0/1812793976 shutdown_connections 2024-03-24T02:52:32.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:32.546+0000 7f6d1adab700 1 -- 172.21.15.17:0/1812793976 wait complete. 2024-03-24T02:52:33.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: mon.b calling monitor election 2024-03-24T02:52:33.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: Updating smithi017:/etc/ceph/ceph.conf 2024-03-24T02:52:33.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-24T02:52:33.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: Updating smithi188:/etc/ceph/ceph.conf 2024-03-24T02:52:33.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:33.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: Updating smithi017:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:33.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:33.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: mon.b calling monitor election 2024-03-24T02:52:33.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: mon.c calling monitor election 2024-03-24T02:52:33.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: mon.a calling monitor election 2024-03-24T02:52:33.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T02:52:33.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T02:52:33.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: fsmap 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: mgrmap e13: x(active, since 61s) 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: Cluster is now healthy 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: overall HEALTH_OK 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1812793976' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:33.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:33.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: mon.b calling monitor election 2024-03-24T02:52:33.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: Updating smithi017:/etc/ceph/ceph.conf 2024-03-24T02:52:33.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-24T02:52:33.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: Updating smithi188:/etc/ceph/ceph.conf 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: Updating smithi017:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: mon.b calling monitor election 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: mon.c calling monitor election 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: mon.a calling monitor election 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: fsmap 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: mgrmap e13: x(active, since 61s) 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: Cluster is now healthy 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: overall HEALTH_OK 2024-03-24T02:52:33.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:33.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:33.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T02:52:33.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:33.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1812793976' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:33.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:33.254 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-24T02:52:33.255 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:52:33.255 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.conf 2024-03-24T02:52:33.284 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:52:33.284 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: mon.b calling monitor election 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: Updating smithi017:/etc/ceph/ceph.conf 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: Updating smithi188:/etc/ceph/ceph.conf 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: Updating smithi017:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: mon.b calling monitor election 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: mon.c calling monitor election 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: mon.a calling monitor election 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0],c=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T02:52:33.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: fsmap 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: mgrmap e13: x(active, since 61s) 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: Cluster is now healthy 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: overall HEALTH_OK 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1812793976' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:33.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:52:33.353 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-24T02:52:33.353 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-03-24T02:52:33.393 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-24T02:52:33.393 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T02:52:33.469 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-24T02:52:33.469 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.conf 2024-03-24T02:52:33.503 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-24T02:52:33.503 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T02:52:33.574 INFO:tasks.cephadm:Adding mgr.x on smithi017 2024-03-24T02:52:33.574 INFO:tasks.cephadm:Adding mgr.y on smithi019 2024-03-24T02:52:33.574 INFO:tasks.cephadm:Adding mgr.z on smithi188 2024-03-24T02:52:33.574 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch apply mgr '3;smithi017=x;smithi019=y;smithi188=z' 2024-03-24T02:52:33.848 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T02:52:34.141 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:33 smithi188 ceph-mon[36206]: Reconfiguring mon.a (unknown last config time)... 2024-03-24T02:52:34.141 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:33 smithi188 ceph-mon[36206]: Reconfiguring daemon mon.a on smithi017 2024-03-24T02:52:34.190 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:33 smithi019 ceph-mon[35944]: Reconfiguring mon.a (unknown last config time)... 2024-03-24T02:52:34.190 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:33 smithi019 ceph-mon[35944]: Reconfiguring daemon mon.a on smithi017 2024-03-24T02:52:34.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:33 smithi017 ceph-mon[31926]: Reconfiguring mon.a (unknown last config time)... 2024-03-24T02:52:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:33 smithi017 ceph-mon[31926]: Reconfiguring daemon mon.a on smithi017 2024-03-24T02:52:34.206 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:52:33 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:52:33.775+0000 7f5f0f6a2700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-24T02:52:34.905 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:34 smithi019 ceph-mon[35944]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:34.905 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:34 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:34.905 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:34 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:34.905 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:34 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:34.905 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:34 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T02:52:34.905 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:34 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:35.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:34 smithi188 ceph-mon[36206]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:35.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:34 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:35.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:34 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:35.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:34 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:35.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:34 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T02:52:35.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:34 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:35.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:34 smithi017 ceph-mon[31926]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:35.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:34 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:35.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:34 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:35.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:34 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:35.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:34 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T02:52:35.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:34 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:35.389 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.324+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2667687754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c68100bf0 msgr2=0x7f1c68101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:35.389 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.324+0000 7f1c6f31e700 1 --2- 172.21.15.188:0/2667687754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c68100bf0 0x7f1c68101010 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009910 tx=0x7f1c64009c20 comp rx=0 tx=0).stop 2024-03-24T02:52:35.389 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.324+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2667687754 shutdown_connections 2024-03-24T02:52:35.389 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.324+0000 7f1c6f31e700 1 --2- 172.21.15.188:0/2667687754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c68100bf0 0x7f1c68101010 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:35.389 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.324+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2667687754 >> 172.21.15.188:0/2667687754 conn(0x7f1c680fc170 msgr2=0x7f1c680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:35.389 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.324+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2667687754 shutdown_connections 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.324+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2667687754 wait complete. 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6f31e700 1 Processor -- start 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6f31e700 1 -- start start 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6f31e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c68100bf0 0x7f1c6806aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6f31e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6819a240 0x7f1c6819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6f31e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6819cbc0 0x7f1c6819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6f31e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68106c30 con 0x7f1c68100bf0 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6f31e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68106ab0 con 0x7f1c6819cbc0 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6f31e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68106db0 con 0x7f1c6819a240 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6d0ba700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c68100bf0 0x7f1c6806aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6d0ba700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c68100bf0 0x7f1c6806aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.188:35530/0 (socket says 172.21.15.188:35530) 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6d0ba700 1 -- 172.21.15.188:0/2788378743 learned_addr learned my addr 172.21.15.188:0/2788378743 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6c8b9700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6819a240 0x7f1c6819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6d8bb700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6819cbc0 0x7f1c6819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6d0ba700 1 -- 172.21.15.188:0/2788378743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6819a240 msgr2=0x7f1c6819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6d0ba700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6819a240 0x7f1c6819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6d0ba700 1 -- 172.21.15.188:0/2788378743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6819cbc0 msgr2=0x7f1c6819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6d0ba700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6819cbc0 0x7f1c6819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6d0ba700 1 -- 172.21.15.188:0/2788378743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6819f930 con 0x7f1c68100bf0 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6d8bb700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6819cbc0 0x7f1c6819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.325+0000 7f1c6c8b9700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6819a240 0x7f1c6819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.326+0000 7f1c6d0ba700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c68100bf0 0x7f1c6806aef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1c64000c00 tx=0x7f1c64005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.326+0000 7f1c5e7fc700 1 -- 172.21.15.188:0/2788378743 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c64027070 con 0x7f1c68100bf0 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.326+0000 7f1c5e7fc700 1 -- 172.21.15.188:0/2788378743 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c64022470 con 0x7f1c68100bf0 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.326+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2788378743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6819fbc0 con 0x7f1c68100bf0 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.326+0000 7f1c5e7fc700 1 -- 172.21.15.188:0/2788378743 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6400f680 con 0x7f1c68100bf0 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.326+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2788378743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c681a00a0 con 0x7f1c68100bf0 2024-03-24T02:52:35.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.327+0000 7f1c5e7fc700 1 -- 172.21.15.188:0/2788378743 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f1c6402a070 con 0x7f1c68100bf0 2024-03-24T02:52:35.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.327+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2788378743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c680637c0 con 0x7f1c68100bf0 2024-03-24T02:52:35.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.327+0000 7f1c5e7fc700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1c54039450 0x7f1c5403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:35.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.327+0000 7f1c5e7fc700 1 -- 172.21.15.188:0/2788378743 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1c6404d220 con 0x7f1c68100bf0 2024-03-24T02:52:35.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.327+0000 7f1c6c8b9700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1c54039450 0x7f1c5403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:35.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.328+0000 7f1c6c8b9700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1c54039450 0x7f1c5403b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1c6806b450 tx=0x7f1c58009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:35.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.333+0000 7f1c5e7fc700 1 -- 172.21.15.188:0/2788378743 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1c64013070 con 0x7f1c68100bf0 2024-03-24T02:52:35.536 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.536+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2788378743 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi017=x;smithi019=y;smithi188=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c68062660 con 0x7f1c54039450 2024-03-24T02:52:35.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.542+0000 7f1c5e7fc700 1 -- 172.21.15.188:0/2788378743 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1c68062660 con 0x7f1c54039450 2024-03-24T02:52:35.542 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mgr update... 2024-03-24T02:52:35.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.544+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2788378743 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1c54039450 msgr2=0x7f1c5403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:35.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.544+0000 7f1c6f31e700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1c54039450 0x7f1c5403b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1c6806b450 tx=0x7f1c58009210 comp rx=0 tx=0).stop 2024-03-24T02:52:35.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.544+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2788378743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c68100bf0 msgr2=0x7f1c6806aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:35.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.544+0000 7f1c6f31e700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c68100bf0 0x7f1c6806aef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1c64000c00 tx=0x7f1c64005000 comp rx=0 tx=0).stop 2024-03-24T02:52:35.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.544+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2788378743 shutdown_connections 2024-03-24T02:52:35.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.544+0000 7f1c6f31e700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1c54039450 0x7f1c5403b910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:35.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.544+0000 7f1c6f31e700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6819cbc0 0x7f1c6819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:35.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.544+0000 7f1c6f31e700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c68100bf0 0x7f1c6806aef0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:35.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.544+0000 7f1c6f31e700 1 --2- 172.21.15.188:0/2788378743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6819a240 0x7f1c6819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:35.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.544+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2788378743 >> 172.21.15.188:0/2788378743 conn(0x7f1c680fc170 msgr2=0x7f1c680fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:35.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.544+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2788378743 shutdown_connections 2024-03-24T02:52:35.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:52:35.544+0000 7f1c6f31e700 1 -- 172.21.15.188:0/2788378743 wait complete. 2024-03-24T02:52:36.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:35 smithi188 ceph-mon[36206]: Reconfiguring mon.b (monmap changed)... 2024-03-24T02:52:36.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:35 smithi188 ceph-mon[36206]: Reconfiguring daemon mon.b on smithi019 2024-03-24T02:52:36.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:36.107 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:36.108 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:36.108 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:36.108 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T02:52:36.108 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:36.109 DEBUG:teuthology.orchestra.run.smithi019:mgr.y> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.y.service 2024-03-24T02:52:36.112 DEBUG:teuthology.orchestra.run.smithi188:mgr.z> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.z.service 2024-03-24T02:52:36.114 INFO:tasks.cephadm:Deploying OSDs... 2024-03-24T02:52:36.114 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:52:36.114 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/scratch_devs of=/dev/stdout 2024-03-24T02:52:36.133 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-24T02:52:36.133 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_1 2024-03-24T02:52:36.142 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:35 smithi019 ceph-mon[35944]: Reconfiguring mon.b (monmap changed)... 2024-03-24T02:52:36.142 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:35 smithi019 ceph-mon[35944]: Reconfiguring daemon mon.b on smithi019 2024-03-24T02:52:36.142 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:36.143 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:36.143 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:36.143 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:36.143 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T02:52:36.143 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:36.192 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-24T02:52:36.192 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T02:52:36.192 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-24T02:52:36.192 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T02:52:36.192 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T02:52:36.192 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-24 02:51:42.127094466 +0000 2024-03-24T02:52:36.192 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-24 02:46:30.229668861 +0000 2024-03-24T02:52:36.192 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-24 02:46:30.229668861 +0000 2024-03-24T02:52:36.192 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-03-24 02:46:30.229668861 +0000 2024-03-24T02:52:36.193 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-24T02:52:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:35 smithi017 ceph-mon[31926]: Reconfiguring mon.b (monmap changed)... 2024-03-24T02:52:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:35 smithi017 ceph-mon[31926]: Reconfiguring daemon mon.b on smithi019 2024-03-24T02:52:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T02:52:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T02:52:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:36.235 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-03-24T02:52:36.235 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-03-24T02:52:36.235 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000114176 s, 4.5 MB/s 2024-03-24T02:52:36.237 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-24T02:52:36.294 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_2 2024-03-24T02:52:36.350 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-24T02:52:36.350 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T02:52:36.350 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-24T02:52:36.350 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T02:52:36.350 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T02:52:36.350 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-24 02:51:42.164094042 +0000 2024-03-24T02:52:36.350 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-24 02:46:30.566665003 +0000 2024-03-24T02:52:36.350 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-24 02:46:30.566665003 +0000 2024-03-24T02:52:36.351 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-03-24 02:46:30.566665003 +0000 2024-03-24T02:52:36.351 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-24T02:52:36.417 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-03-24T02:52:36.417 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-03-24T02:52:36.417 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000111515 s, 4.6 MB/s 2024-03-24T02:52:36.418 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-24T02:52:36.475 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_3 2024-03-24T02:52:36.530 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-24T02:52:36.530 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T02:52:36.531 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-24T02:52:36.531 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T02:52:36.531 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T02:52:36.531 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-24 02:51:42.197093664 +0000 2024-03-24T02:52:36.531 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-24 02:46:30.899661191 +0000 2024-03-24T02:52:36.531 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-24 02:46:30.899661191 +0000 2024-03-24T02:52:36.531 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-03-24 02:46:30.899661191 +0000 2024-03-24T02:52:36.531 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-24T02:52:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-03-24T02:52:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-03-24T02:52:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 8.8076e-05 s, 5.8 MB/s 2024-03-24T02:52:36.598 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-24T02:52:36.653 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_4 2024-03-24T02:52:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-24T02:52:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T02:52:36.709 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-24T02:52:36.709 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T02:52:36.709 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T02:52:36.709 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-24 02:51:42.228093308 +0000 2024-03-24T02:52:36.709 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-24 02:46:31.238657311 +0000 2024-03-24T02:52:36.709 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-24 02:46:31.238657311 +0000 2024-03-24T02:52:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-03-24 02:46:31.238657311 +0000 2024-03-24T02:52:36.709 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-24T02:52:36.774 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-03-24T02:52:36.775 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-03-24T02:52:36.775 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000108557 s, 4.7 MB/s 2024-03-24T02:52:36.776 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-24T02:52:36.831 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-24T02:52:36.832 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-03-24T02:52:36.851 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-24T02:52:36.851 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-03-24T02:52:36.909 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-24T02:52:36.909 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T02:52:36.909 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-03-24T02:52:36.909 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T02:52:36.909 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T02:52:36.909 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-03-24 02:52:24.208669832 +0000 2024-03-24T02:52:36.909 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-24 02:46:57.345126894 +0000 2024-03-24T02:52:36.910 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-24 02:46:57.345126894 +0000 2024-03-24T02:52:36.910 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-24 02:46:57.345126894 +0000 2024-03-24T02:52:36.910 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-24T02:52:36.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-03-24T02:52:36.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-03-24T02:52:36.984 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000270436 s, 1.9 MB/s 2024-03-24T02:52:36.986 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-24T02:52:37.044 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-03-24T02:52:37.102 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-24T02:52:37.102 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T02:52:37.102 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-03-24T02:52:37.102 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T02:52:37.102 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T02:52:37.102 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-03-24 02:52:24.208669832 +0000 2024-03-24T02:52:37.103 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-24 02:46:57.693123864 +0000 2024-03-24T02:52:37.103 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-24 02:46:57.693123864 +0000 2024-03-24T02:52:37.103 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-24 02:46:57.693123864 +0000 2024-03-24T02:52:37.103 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-24T02:52:37.173 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:36 smithi019 ceph-mon[35944]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi017=x;smithi019=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:52:37.173 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:36 smithi019 ceph-mon[35944]: Saving service mgr spec with placement smithi017=x;smithi019=y;smithi188=z;count:3 2024-03-24T02:52:37.173 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:36 smithi019 ceph-mon[35944]: Reconfiguring mon.c (monmap changed)... 2024-03-24T02:52:37.173 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:36 smithi019 ceph-mon[35944]: Reconfiguring daemon mon.c on smithi188 2024-03-24T02:52:37.173 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:36 smithi019 ceph-mon[35944]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-03-24T02:52:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-03-24T02:52:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000226018 s, 2.3 MB/s 2024-03-24T02:52:37.177 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-24T02:52:37.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:36 smithi017 ceph-mon[31926]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi017=x;smithi019=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:52:37.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:36 smithi017 ceph-mon[31926]: Saving service mgr spec with placement smithi017=x;smithi019=y;smithi188=z;count:3 2024-03-24T02:52:37.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:36 smithi017 ceph-mon[31926]: Reconfiguring mon.c (monmap changed)... 2024-03-24T02:52:37.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:36 smithi017 ceph-mon[31926]: Reconfiguring daemon mon.c on smithi188 2024-03-24T02:52:37.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:36 smithi017 ceph-mon[31926]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:37.235 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-03-24T02:52:37.292 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-24T02:52:37.292 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T02:52:37.292 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-24T02:52:37.292 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T02:52:37.292 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T02:52:37.292 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-03-24 02:52:24.209669824 +0000 2024-03-24T02:52:37.292 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-24 02:46:58.055120711 +0000 2024-03-24T02:52:37.292 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-24 02:46:58.055120711 +0000 2024-03-24T02:52:37.292 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-24 02:46:58.055120711 +0000 2024-03-24T02:52:37.293 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-24T02:52:37.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:36 smithi188 ceph-mon[36206]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi017=x;smithi019=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:52:37.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:36 smithi188 ceph-mon[36206]: Saving service mgr spec with placement smithi017=x;smithi019=y;smithi188=z;count:3 2024-03-24T02:52:37.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:36 smithi188 ceph-mon[36206]: Reconfiguring mon.c (monmap changed)... 2024-03-24T02:52:37.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:36 smithi188 ceph-mon[36206]: Reconfiguring daemon mon.c on smithi188 2024-03-24T02:52:37.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:36 smithi188 ceph-mon[36206]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:37.363 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-03-24T02:52:37.364 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-03-24T02:52:37.364 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000198572 s, 2.6 MB/s 2024-03-24T02:52:37.365 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-24T02:52:37.422 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-03-24T02:52:37.479 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-24T02:52:37.479 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T02:52:37.479 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-24T02:52:37.479 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T02:52:37.479 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T02:52:37.479 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-03-24 02:52:24.209669824 +0000 2024-03-24T02:52:37.479 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-24 02:46:58.400117706 +0000 2024-03-24T02:52:37.479 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-24 02:46:58.400117706 +0000 2024-03-24T02:52:37.479 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-24 02:46:58.400117706 +0000 2024-03-24T02:52:37.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-24T02:52:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-03-24T02:52:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-03-24T02:52:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000220845 s, 2.3 MB/s 2024-03-24T02:52:37.552 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-24T02:52:37.610 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-24T02:52:37.610 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-03-24T02:52:37.626 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-24T02:52:37.627 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-03-24T02:52:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-24T02:52:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T02:52:37.681 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-24T02:52:37.682 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T02:52:37.682 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T02:52:37.682 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-03-24 02:52:14.911144942 +0000 2024-03-24T02:52:37.682 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-03-24 02:46:49.681478969 +0000 2024-03-24T02:52:37.682 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-03-24 02:46:49.681478969 +0000 2024-03-24T02:52:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-03-24 02:46:49.681478969 +0000 2024-03-24T02:52:37.682 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-24T02:52:37.748 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-03-24T02:52:37.749 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-03-24T02:52:37.749 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000342545 s, 1.5 MB/s 2024-03-24T02:52:37.750 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-24T02:52:37.806 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-03-24T02:52:37.861 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-24T02:52:37.861 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T02:52:37.861 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-24T02:52:37.861 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T02:52:37.861 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T02:52:37.861 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-03-24 02:52:14.911144942 +0000 2024-03-24T02:52:37.861 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-03-24 02:46:50.053472570 +0000 2024-03-24T02:52:37.861 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-03-24 02:46:50.053472570 +0000 2024-03-24T02:52:37.862 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-03-24 02:46:50.053472570 +0000 2024-03-24T02:52:37.862 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-24T02:52:37.927 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-03-24T02:52:37.928 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-03-24T02:52:37.928 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000205365 s, 2.5 MB/s 2024-03-24T02:52:37.929 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-24T02:52:37.985 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-03-24T02:52:38.039 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-24T02:52:38.039 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T02:52:38.039 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-24T02:52:38.039 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T02:52:38.039 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T02:52:38.039 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-03-24 02:52:14.912144925 +0000 2024-03-24T02:52:38.039 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-03-24 02:46:50.391466755 +0000 2024-03-24T02:52:38.039 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-03-24 02:46:50.391466755 +0000 2024-03-24T02:52:38.039 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-03-24 02:46:50.391466755 +0000 2024-03-24T02:52:38.040 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-24T02:52:38.105 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-03-24T02:52:38.105 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-03-24T02:52:38.105 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000200176 s, 2.6 MB/s 2024-03-24T02:52:38.106 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-24T02:52:38.162 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-03-24T02:52:38.217 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-24T02:52:38.217 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T02:52:38.217 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-03-24T02:52:38.217 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T02:52:38.217 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T02:52:38.218 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-03-24 02:52:14.912144925 +0000 2024-03-24T02:52:38.218 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-03-24 02:46:50.724461027 +0000 2024-03-24T02:52:38.218 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-03-24 02:46:50.724461027 +0000 2024-03-24T02:52:38.218 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-03-24 02:46:50.724461027 +0000 2024-03-24T02:52:38.218 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-24T02:52:38.283 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-03-24T02:52:38.283 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-03-24T02:52:38.283 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000181116 s, 2.8 MB/s 2024-03-24T02:52:38.284 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-24T02:52:38.316 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:38.316 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:38.316 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:38.316 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:38.316 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:38.316 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:38.316 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T02:52:38.316 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T02:52:38.316 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T02:52:38.316 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:38.317 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:38 smithi188 ceph-mon[36206]: Deploying daemon mgr.z on smithi188 2024-03-24T02:52:38.317 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:38 smithi188 ceph-mon[36206]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:38.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:38.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:38.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T02:52:38.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T02:52:38.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T02:52:38.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:38.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:38 smithi019 ceph-mon[35944]: Deploying daemon mgr.z on smithi188 2024-03-24T02:52:38.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:38 smithi019 ceph-mon[35944]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:38.331 INFO:tasks.cephadm:Deploying osd.0 on smithi017 with /dev/vg_nvme/lv_4... 2024-03-24T02:52:38.331 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-24T02:52:38.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:38.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:38.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:38.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:38.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:38.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:38.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T02:52:38.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T02:52:38.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T02:52:38.359 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:38.359 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:38 smithi017 ceph-mon[31926]: Deploying daemon mgr.z on smithi188 2024-03-24T02:52:38.359 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:38 smithi017 ceph-mon[31926]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:38.573 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:52:39.732 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:39 smithi188 systemd[1]: Starting Ceph mgr.z for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:52:40.033 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:39 smithi188 ceph-mon[36206]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:40.034 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:39 smithi188 podman[38046]: 2024-03-24 02:52:39.732928193 +0000 UTC m=+0.094817559 container create b80ef274913f6c0f34974588198694a5c62e9378fcf2d10885617d93e5d60dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T02:52:40.034 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:39 smithi188 podman[38046]: 2024-03-24 02:52:39.655937904 +0000 UTC m=+0.017827280 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T02:52:40.034 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:39 smithi188 podman[38046]: 2024-03-24 02:52:39.935078253 +0000 UTC m=+0.296967634 container init b80ef274913f6c0f34974588198694a5c62e9378fcf2d10885617d93e5d60dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T02:52:40.034 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:39 smithi188 podman[38046]: 2024-03-24 02:52:39.937696681 +0000 UTC m=+0.299586047 container start b80ef274913f6c0f34974588198694a5c62e9378fcf2d10885617d93e5d60dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-24T02:52:40.034 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:39 smithi188 ceph-mgr[38065]: -- 172.21.15.188:0/4213467400 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d086b5c5a0 con 0x55d086b30c00 2024-03-24T02:52:40.034 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:40 smithi188 bash[38046]: b80ef274913f6c0f34974588198694a5c62e9378fcf2d10885617d93e5d60dee 2024-03-24T02:52:40.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:39 smithi017 ceph-mon[31926]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:40.297 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:40 smithi188 systemd[1]: Started Ceph mgr.z for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:52:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:39 smithi019 ceph-mon[35944]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-24T02:52:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-24T02:52:40.322 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2024-03-24T02:52:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2024-03-24T02:52:40.323 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00829542 s, 1.3 GB/s 2024-03-24T02:52:40.323 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2024-03-24T02:52:40.592 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:40 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:40.497+0000 7feab859e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T02:52:41.007 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi017:vg_nvme/lv_4 2024-03-24T02:52:41.047 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:40 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:40.593+0000 7feab859e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T02:52:41.290 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:52:41.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:41.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:41.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:41.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T02:52:41.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T02:52:41.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T02:52:41.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:41.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:41 smithi017 ceph-mon[31926]: Deploying daemon mgr.y on smithi019 2024-03-24T02:52:41.472 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:41.472 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:41.472 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:41.472 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T02:52:41.472 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T02:52:41.472 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T02:52:41.472 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:41.472 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-mon[36206]: Deploying daemon mgr.y on smithi019 2024-03-24T02:52:41.472 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:41.240+0000 7feab859e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T02:52:41.473 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:41.332+0000 7feab859e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T02:52:41.473 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:41.472+0000 7feab859e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T02:52:41.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:41.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:41.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:41.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T02:52:41.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T02:52:41.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T02:52:41.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:41.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:41 smithi019 ceph-mon[35944]: Deploying daemon mgr.y on smithi019 2024-03-24T02:52:41.797 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:41.567+0000 7feab859e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T02:52:41.798 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:41.648+0000 7feab859e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T02:52:42.187 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:42 smithi188 ceph-mon[36206]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:42.188 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:41 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:41.906+0000 7feab859e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T02:52:42.188 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:42 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:42.012+0000 7feab859e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T02:52:42.188 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:42 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:42.095+0000 7feab859e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T02:52:42.188 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:42 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:42.187+0000 7feab859e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T02:52:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:42 smithi019 ceph-mon[35944]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:42.320 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:42 smithi019 systemd[1]: Starting Ceph mgr.y for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:52:42.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:42 smithi017 ceph-mon[31926]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:42.547 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:42 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:42.361+0000 7feab859e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T02:52:42.638 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:42 smithi019 podman[37314]: 2024-03-24 02:52:42.279684334 +0000 UTC m=+0.018522696 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T02:52:42.638 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:42 smithi019 podman[37314]: 2024-03-24 02:52:42.409505912 +0000 UTC m=+0.148344258 container create 283658810b5c98dc8ee84f381c73e1d759f72e4e9c45ad5338662386d1aa557d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-24T02:52:42.638 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:42 smithi019 podman[37314]: 2024-03-24 02:52:42.61177321 +0000 UTC m=+0.350611548 container init 283658810b5c98dc8ee84f381c73e1d759f72e4e9c45ad5338662386d1aa557d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-24T02:52:42.638 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:42 smithi019 podman[37314]: 2024-03-24 02:52:42.614529963 +0000 UTC m=+0.353368321 container start 283658810b5c98dc8ee84f381c73e1d759f72e4e9c45ad5338662386d1aa557d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-24T02:52:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.793+0000 7f5fae327700 1 -- 172.21.15.17:0/3601323004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fa8102ca0 msgr2=0x7f5fa8105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.793+0000 7f5fae327700 1 --2- 172.21.15.17:0/3601323004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fa8102ca0 0x7f5fa8105090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009990 tx=0x7f5f98009ca0 comp rx=0 tx=0).stop 2024-03-24T02:52:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.793+0000 7f5fae327700 1 -- 172.21.15.17:0/3601323004 shutdown_connections 2024-03-24T02:52:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.793+0000 7f5fae327700 1 --2- 172.21.15.17:0/3601323004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fa8107f00 0x7f5fa810a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.793+0000 7f5fae327700 1 --2- 172.21.15.17:0/3601323004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fa8102ca0 0x7f5fa8105090 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.793+0000 7f5fae327700 1 --2- 172.21.15.17:0/3601323004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fa81055d0 0x7f5fa81079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.793+0000 7f5fae327700 1 -- 172.21.15.17:0/3601323004 >> 172.21.15.17:0/3601323004 conn(0x7f5fa80fc650 msgr2=0x7f5fa80fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:52:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.793+0000 7f5fae327700 1 -- 172.21.15.17:0/3601323004 shutdown_connections 2024-03-24T02:52:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.793+0000 7f5fae327700 1 -- 172.21.15.17:0/3601323004 wait complete. 2024-03-24T02:52:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fae327700 1 Processor -- start 2024-03-24T02:52:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fae327700 1 -- start start 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fae327700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fa8102ca0 0x7f5fa8102600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fae327700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fa81055d0 0x7f5fa8100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fae327700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fa8107f00 0x7f5fa8101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fae327700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa810de10 con 0x7f5fa8107f00 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fae327700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa810dc90 con 0x7f5fa81055d0 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fae327700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa810df90 con 0x7f5fa8102ca0 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fac8c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fa8107f00 0x7f5fa8101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fa77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fa81055d0 0x7f5fa8100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fa77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fa81055d0 0x7f5fa8100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:33158/0 (socket says 172.21.15.17:33158) 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fac8c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fa8107f00 0x7f5fa8101190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40164/0 (socket says 172.21.15.17:40164) 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fa77fe700 1 -- 172.21.15.17:0/3695738524 learned_addr learned my addr 172.21.15.17:0/3695738524 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fa7fff700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fa8102ca0 0x7f5fa8102600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:42.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fa77fe700 1 -- 172.21.15.17:0/3695738524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fa8102ca0 msgr2=0x7f5fa8102600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fa77fe700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fa8102ca0 0x7f5fa8102600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fa77fe700 1 -- 172.21.15.17:0/3695738524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fa8107f00 msgr2=0x7f5fa8101190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fa77fe700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fa8107f00 0x7f5fa8101190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fa77fe700 1 -- 172.21.15.17:0/3695738524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa8101a40 con 0x7f5fa81055d0 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.794+0000 7f5fac8c4700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fa8107f00 0x7f5fa8101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.795+0000 7f5fa77fe700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fa81055d0 0x7f5fa8100c50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00e9a0 tx=0x7f5f9c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.795+0000 7f5fa57fa700 1 -- 172.21.15.17:0/3695738524 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c00ca10 con 0x7f5fa81055d0 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.795+0000 7f5fae327700 1 -- 172.21.15.17:0/3695738524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa81a2590 con 0x7f5fa81055d0 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.795+0000 7f5fa57fa700 1 -- 172.21.15.17:0/3695738524 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f9c00cbb0 con 0x7f5fa81055d0 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.795+0000 7f5fa57fa700 1 -- 172.21.15.17:0/3695738524 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c018680 con 0x7f5fa81055d0 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.795+0000 7f5fae327700 1 -- 172.21.15.17:0/3695738524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa81a2ad0 con 0x7f5fa81055d0 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.796+0000 7f5fa57fa700 1 -- 172.21.15.17:0/3695738524 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f5f9c010430 con 0x7f5fa81055d0 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.796+0000 7f5fae327700 1 -- 172.21.15.17:0/3695738524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa8068e50 con 0x7f5fa81055d0 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.796+0000 7f5fa57fa700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5f90039200 0x7f5f9003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.796+0000 7f5fa57fa700 1 -- 172.21.15.17:0/3695738524 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5f9c04c350 con 0x7f5fa81055d0 2024-03-24T02:52:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.796+0000 7f5fa7fff700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5f90039200 0x7f5f9003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:52:42.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.797+0000 7f5fa7fff700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5f90039200 0x7f5f9003b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800b2e0 tx=0x7f5f98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:52:42.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:42.806+0000 7f5fa57fa700 1 -- 172.21.15.17:0/3695738524 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5f9c018960 con 0x7f5fa81055d0 2024-03-24T02:52:43.019 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:42 smithi019 ceph-mgr[37333]: -- 172.21.15.19:0/2379272097 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b94d87a5a0 con 0x55b94d84f000 2024-03-24T02:52:43.019 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:42 smithi019 bash[37314]: 283658810b5c98dc8ee84f381c73e1d759f72e4e9c45ad5338662386d1aa557d 2024-03-24T02:52:43.019 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:42 smithi019 systemd[1]: Started Ceph mgr.y for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:52:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:43.023+0000 7f5fae327700 1 -- 172.21.15.17:0/3695738524 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fa80625d0 con 0x7f5f90039200 2024-03-24T02:52:43.287 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:43 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:43.196+0000 7f8756395200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T02:52:43.570 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:43 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:43.299+0000 7f8756395200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T02:52:43.762 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:43 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:43.337+0000 7feab859e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T02:52:43.762 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:43 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:43.482+0000 7feab859e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T02:52:43.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:43 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:43.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:43 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:43.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:43 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:43.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:43 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:43.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:43 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:43.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:43 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:52:43.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:43 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:52:43.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:43 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:44.018 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:43 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.018 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:43 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.018 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:43 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.018 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:43 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.018 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:43 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:44.018 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:43 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:52:44.018 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:43 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:52:44.018 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:43 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:44.047 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:43 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:43.848+0000 7feab859e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T02:52:44.048 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:43 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:43.942+0000 7feab859e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T02:52:44.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:43 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:43 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:43 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:43 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:43 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:44.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:43 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:52:44.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:43 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:52:44.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:43 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:44.306 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:44.017+0000 7f8756395200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T02:52:44.306 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:44.115+0000 7f8756395200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T02:52:44.306 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:44.267+0000 7f8756395200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T02:52:44.547 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:44.137+0000 7feab859e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T02:52:44.548 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:44.334+0000 7feab859e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T02:52:44.570 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:44.371+0000 7f8756395200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T02:52:44.570 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:44.459+0000 7f8756395200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T02:52:44.897 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:44.620+0000 7feab859e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T02:52:44.898 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-mon[36206]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:52:44.898 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-mon[36206]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:44.898 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.898 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.898 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.898 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.898 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:44.898 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:44.898 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.898 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T02:52:44.898 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T02:52:44.899 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:44.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:44 smithi017 ceph-mon[31926]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:52:44.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:44 smithi017 ceph-mon[31926]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:44.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:44.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:44.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:44.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T02:52:44.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T02:52:44.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:45.034 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:44.733+0000 7f8756395200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T02:52:45.034 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:44.847+0000 7f8756395200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T02:52:45.034 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:44.935+0000 7f8756395200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T02:52:45.034 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:45 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:45.034+0000 7f8756395200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T02:52:45.034 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-mon[35944]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:52:45.034 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-mon[35944]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:45.034 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:45.034 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:45.035 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:45.035 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:45.035 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:45.035 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:45.035 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:45.035 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T02:52:45.035 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T02:52:45.035 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:45.297 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:44 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:44.897+0000 7feab859e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T02:52:45.298 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:45 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:45.132+0000 7feab859e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T02:52:45.320 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:45 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:45.216+0000 7f8756395200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T02:52:46.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:45 smithi188 ceph-mon[36206]: Reconfiguring mgr.x (unknown last config time)... 2024-03-24T02:52:46.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:45 smithi188 ceph-mon[36206]: Reconfiguring daemon mgr.x on smithi017 2024-03-24T02:52:46.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:46.048 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:45 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:45.930+0000 7feab859e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T02:52:46.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:45 smithi019 ceph-mon[35944]: Reconfiguring mgr.x (unknown last config time)... 2024-03-24T02:52:46.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:45 smithi019 ceph-mon[35944]: Reconfiguring daemon mgr.x on smithi017 2024-03-24T02:52:46.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:46.122 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:45 smithi017 ceph-mon[31926]: Reconfiguring mgr.x (unknown last config time)... 2024-03-24T02:52:46.122 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:45 smithi017 ceph-mon[31926]: Reconfiguring daemon mgr.x on smithi017 2024-03-24T02:52:46.122 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:46.570 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:46 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:46.247+0000 7f8756395200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T02:52:46.570 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:46 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:46.383+0000 7f8756395200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T02:52:46.768 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:46 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:46.502+0000 7feab859e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T02:52:46.768 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:46 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:46.599+0000 7feab859e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T02:52:47.022 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:46 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:46.728+0000 7f8756395200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T02:52:47.022 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:46 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:46.821+0000 7f8756395200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T02:52:47.022 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:47 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:47.022+0000 7f8756395200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T02:52:47.022 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:46 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4173885135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07cdbc7a-3ba5-4284-8819-b9352e15e8c8"}]: dispatch 2024-03-24T02:52:47.023 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:46 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4173885135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07cdbc7a-3ba5-4284-8819-b9352e15e8c8"}]': finished 2024-03-24T02:52:47.023 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:46 smithi019 ceph-mon[35944]: osdmap e5: 1 total, 0 up, 1 in 2024-03-24T02:52:47.023 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:46 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:52:47.023 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:46 smithi019 ceph-mon[35944]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:47.023 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:46 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3847476951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:52:47.047 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 02:52:46 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T02:52:46.816+0000 7feab859e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T02:52:47.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:46 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4173885135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07cdbc7a-3ba5-4284-8819-b9352e15e8c8"}]: dispatch 2024-03-24T02:52:47.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:46 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4173885135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07cdbc7a-3ba5-4284-8819-b9352e15e8c8"}]': finished 2024-03-24T02:52:47.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:46 smithi188 ceph-mon[36206]: osdmap e5: 1 total, 0 up, 1 in 2024-03-24T02:52:47.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:46 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:52:47.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:46 smithi188 ceph-mon[36206]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:47.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:46 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3847476951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:52:47.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:46 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4173885135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07cdbc7a-3ba5-4284-8819-b9352e15e8c8"}]: dispatch 2024-03-24T02:52:47.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:46 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4173885135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07cdbc7a-3ba5-4284-8819-b9352e15e8c8"}]': finished 2024-03-24T02:52:47.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:46 smithi017 ceph-mon[31926]: osdmap e5: 1 total, 0 up, 1 in 2024-03-24T02:52:47.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:46 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:52:47.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:46 smithi017 ceph-mon[31926]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:47.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:46 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3847476951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:52:47.320 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:47 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:47.212+0000 7f8756395200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T02:52:47.772 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:47 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:47.504+0000 7f8756395200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T02:52:47.772 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:47 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:47.772+0000 7f8756395200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T02:52:47.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:47.801+0000 7f5fa57fa700 1 -- 172.21.15.17:0/3695738524 <== mon.2 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 14) v1 ==== 91698+0+0 (secure 0 0 0) 0x7f5f9c010720 con 0x7f5fa81055d0 2024-03-24T02:52:48.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:47 smithi188 ceph-mon[36206]: Standby manager daemon z started 2024-03-24T02:52:48.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:47 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.188:0/50702962' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-24T02:52:48.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:47 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.188:0/50702962' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T02:52:48.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:47 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.188:0/50702962' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-24T02:52:48.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:47 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.188:0/50702962' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T02:52:48.070 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:48 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:48.016+0000 7f8756395200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T02:52:48.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:47 smithi019 ceph-mon[35944]: Standby manager daemon z started 2024-03-24T02:52:48.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:47 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.188:0/50702962' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-24T02:52:48.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:47 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.188:0/50702962' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T02:52:48.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:47 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.188:0/50702962' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-24T02:52:48.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:47 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.188:0/50702962' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T02:52:48.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:47 smithi017 ceph-mon[31926]: Standby manager daemon z started 2024-03-24T02:52:48.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:47 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.188:0/50702962' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-24T02:52:48.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:47 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.188:0/50702962' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T02:52:48.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:47 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.188:0/50702962' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-24T02:52:48.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:47 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.188:0/50702962' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T02:52:49.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:48 smithi019 ceph-mon[35944]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-24T02:52:49.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:48 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-24T02:52:49.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:48 smithi019 ceph-mon[35944]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:49.071 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:48 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:48.851+0000 7f8756395200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T02:52:49.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:48 smithi017 ceph-mon[31926]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-24T02:52:49.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:48 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-24T02:52:49.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:48 smithi017 ceph-mon[31926]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:49.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:48 smithi188 ceph-mon[36206]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-24T02:52:49.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:48 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-24T02:52:49.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:48 smithi188 ceph-mon[36206]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:49.766 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:49 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:49.444+0000 7f8756395200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T02:52:49.766 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:49 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:49.544+0000 7f8756395200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T02:52:49.766 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 02:52:49 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T02:52:49.766+0000 7f8756395200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T02:52:49.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:52:49.813+0000 7f5fa57fa700 1 -- 172.21.15.17:0/3695738524 <== mon.2 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f5f9c017bf0 con 0x7f5fa81055d0 2024-03-24T02:52:50.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:49 smithi019 ceph-mon[35944]: Standby manager daemon y started 2024-03-24T02:52:50.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:49 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.19:0/4239015239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T02:52:50.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:49 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.19:0/4239015239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T02:52:50.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:49 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.19:0/4239015239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T02:52:50.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:49 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.19:0/4239015239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T02:52:50.157 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:49 smithi017 ceph-mon[31926]: Standby manager daemon y started 2024-03-24T02:52:50.157 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:49 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.19:0/4239015239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T02:52:50.157 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:49 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.19:0/4239015239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T02:52:50.157 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:49 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.19:0/4239015239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T02:52:50.157 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:49 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.19:0/4239015239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T02:52:50.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:49 smithi188 ceph-mon[36206]: Standby manager daemon y started 2024-03-24T02:52:50.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:49 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.19:0/4239015239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T02:52:50.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:49 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.19:0/4239015239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T02:52:50.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:49 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.19:0/4239015239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T02:52:50.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:49 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.19:0/4239015239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T02:52:51.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:50 smithi019 ceph-mon[35944]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-03-24T02:52:51.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:50 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T02:52:51.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:50 smithi019 ceph-mon[35944]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:51.152 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:50 smithi017 ceph-mon[31926]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-03-24T02:52:51.152 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:50 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T02:52:51.152 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:50 smithi017 ceph-mon[31926]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:51.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:50 smithi188 ceph-mon[36206]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-03-24T02:52:51.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:50 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T02:52:51.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:50 smithi188 ceph-mon[36206]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:52.162 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:52 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:52.163 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:52 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:52.163 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:52 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:52.163 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:52 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:52.163 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:52 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:52.163 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:52 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:52.163 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:52 smithi017 ceph-mon[31926]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:52 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:52 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:52 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:52 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:52.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:52 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:52.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:52 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:52.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:52 smithi019 ceph-mon[35944]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:52.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:52 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:52.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:52 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:52.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:52 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:52:52.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:52 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:52.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:52 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:52:52.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:52 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:52:52.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:52 smithi188 ceph-mon[36206]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:53.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T02:52:53.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:53.769 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T02:52:53.769 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:53.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T02:52:53.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:52:54.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:54 smithi017 ceph-mon[31926]: Deploying daemon osd.0 on smithi017 2024-03-24T02:52:54.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:54 smithi017 ceph-mon[31926]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:54.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:54 smithi188 ceph-mon[36206]: Deploying daemon osd.0 on smithi017 2024-03-24T02:52:54.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:54 smithi188 ceph-mon[36206]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:54.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:54 smithi019 ceph-mon[35944]: Deploying daemon osd.0 on smithi017 2024-03-24T02:52:54.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:54 smithi019 ceph-mon[35944]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:56.105 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:55 smithi017 ceph-mon[31926]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:56.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:55 smithi188 ceph-mon[36206]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:55 smithi019 ceph-mon[35944]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:57.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:57 smithi017 ceph-mon[31926]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:58.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:57 smithi188 ceph-mon[36206]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:52:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:57 smithi019 ceph-mon[35944]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:52:59 smithi017 ceph-mon[31926]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:00.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:59 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:00.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:59 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:00.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:59 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:00.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:52:59 smithi188 ceph-mon[36206]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:00.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:59 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:00.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:59 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:00.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:59 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:00.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:52:59 smithi019 ceph-mon[35944]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:02.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:01 smithi017 ceph-mon[31926]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:02.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:01 smithi188 ceph-mon[36206]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:02.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:01 smithi019 ceph-mon[35944]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:02.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.332+0000 7f5fa57fa700 1 -- 172.21.15.17:0/3695738524 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5fa80625d0 con 0x7f5f90039200 2024-03-24T02:53:02.332 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 0 on host 'smithi017' 2024-03-24T02:53:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.334+0000 7f5fae327700 1 -- 172.21.15.17:0/3695738524 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5f90039200 msgr2=0x7f5f9003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.334+0000 7f5fae327700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5f90039200 0x7f5f9003b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800b2e0 tx=0x7f5f98005fb0 comp rx=0 tx=0).stop 2024-03-24T02:53:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.334+0000 7f5fae327700 1 -- 172.21.15.17:0/3695738524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fa81055d0 msgr2=0x7f5fa8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.334+0000 7f5fae327700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fa81055d0 0x7f5fa8100c50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00e9a0 tx=0x7f5f9c00ecb0 comp rx=0 tx=0).stop 2024-03-24T02:53:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.334+0000 7f5fae327700 1 -- 172.21.15.17:0/3695738524 shutdown_connections 2024-03-24T02:53:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.334+0000 7f5fae327700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5f90039200 0x7f5f9003b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.334+0000 7f5fae327700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fa8107f00 0x7f5fa8101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.334+0000 7f5fae327700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fa8102ca0 0x7f5fa8102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.334+0000 7f5fae327700 1 --2- 172.21.15.17:0/3695738524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fa81055d0 0x7f5fa8100c50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.334+0000 7f5fae327700 1 -- 172.21.15.17:0/3695738524 >> 172.21.15.17:0/3695738524 conn(0x7f5fa80fc650 msgr2=0x7f5fa81036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:53:02.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.334+0000 7f5fae327700 1 -- 172.21.15.17:0/3695738524 shutdown_connections 2024-03-24T02:53:02.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:02.334+0000 7f5fae327700 1 -- 172.21.15.17:0/3695738524 wait complete. 2024-03-24T02:53:02.986 DEBUG:teuthology.orchestra.run.smithi017:osd.0> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.0.service 2024-03-24T02:53:02.989 INFO:tasks.cephadm:Deploying osd.1 on smithi019 with /dev/vg_nvme/lv_4... 2024-03-24T02:53:02.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-24T02:53:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T02:53:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:03 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:03 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:03 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:03 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:03 smithi188 ceph-mon[36206]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T02:53:03.556 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:03.556 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:03.556 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:03.556 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:03.556 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:03 smithi017 ceph-mon[31926]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T02:53:03.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:03 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:03.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:03 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:03.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:03 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:03.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:03 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:03.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:03 smithi019 ceph-mon[35944]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T02:53:04.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:04 smithi019 ceph-mon[35944]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T02:53:04.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:04 smithi019 ceph-mon[35944]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T02:53:04.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:04 smithi019 ceph-mon[35944]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-03-24T02:53:04.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:04 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:04.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:04 smithi019 ceph-mon[35944]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:04.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:04 smithi017 ceph-mon[31926]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T02:53:04.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:04 smithi017 ceph-mon[31926]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T02:53:04.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:04 smithi017 ceph-mon[31926]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-03-24T02:53:04.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:04 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:04.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:04 smithi017 ceph-mon[31926]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:05.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:04 smithi188 ceph-mon[36206]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T02:53:05.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:04 smithi188 ceph-mon[36206]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T02:53:05.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:04 smithi188 ceph-mon[36206]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-03-24T02:53:05.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:04 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:05.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:04 smithi188 ceph-mon[36206]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:05.142 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-24T02:53:05.142 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-24T02:53:05.142 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-03-24T02:53:05.142 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-03-24T02:53:05.142 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111772 s, 938 MB/s 2024-03-24T02:53:05.143 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 2024-03-24T02:53:05.143 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-03-24T02:53:05.617 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 02:53:05 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0[46572]: 2024-03-24T02:53:05.216+0000 7f8f25e7c700 -1 osd.0 0 waiting for initial osdmap 2024-03-24T02:53:05.617 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 02:53:05 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0[46572]: 2024-03-24T02:53:05.220+0000 7f8f1d466700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T02:53:05.785 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:05 smithi019 ceph-mon[35944]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-03-24T02:53:05.785 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:05 smithi019 ceph-mon[35944]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T02:53:05.785 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:05.785 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:05.785 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:05 smithi019 ceph-mon[35944]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' 2024-03-24T02:53:05.813 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-03-24T02:53:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:05 smithi017 ceph-mon[31926]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-03-24T02:53:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:05 smithi017 ceph-mon[31926]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T02:53:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:05 smithi017 ceph-mon[31926]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' 2024-03-24T02:53:06.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:05 smithi188 ceph-mon[36206]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-03-24T02:53:06.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:05 smithi188 ceph-mon[36206]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T02:53:06.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:06.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:06.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:05 smithi188 ceph-mon[36206]: from='osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551]' entity='osd.0' 2024-03-24T02:53:06.063 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T02:53:06.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: purged_snaps scrub starts 2024-03-24T02:53:06.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: purged_snaps scrub ok 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551] boot 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: osdmap e8: 1 total, 1 up, 1 in 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:06.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:06 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:06.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: purged_snaps scrub starts 2024-03-24T02:53:06.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: purged_snaps scrub ok 2024-03-24T02:53:06.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:06.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:06.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:06.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:06.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:53:06.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:06.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:06.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:06.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:06.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551] boot 2024-03-24T02:53:06.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: osdmap e8: 1 total, 1 up, 1 in 2024-03-24T02:53:06.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:06.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:07.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: purged_snaps scrub starts 2024-03-24T02:53:07.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: purged_snaps scrub ok 2024-03-24T02:53:07.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:07.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T02:53:07.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:07.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:07.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:53:07.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:07.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:07.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:07.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:07.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: osd.0 [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551] boot 2024-03-24T02:53:07.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: osdmap e8: 1 total, 1 up, 1 in 2024-03-24T02:53:07.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T02:53:07.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:06 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.579+0000 7ff43922d700 1 -- 172.21.15.19:0/2077650566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4341021c0 msgr2=0x7ff4341025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.579+0000 7ff43922d700 1 --2- 172.21.15.19:0/2077650566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4341021c0 0x7ff4341025e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4240099c0 tx=0x7ff424009cd0 comp rx=0 tx=0).stop 2024-03-24T02:53:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.579+0000 7ff43922d700 1 -- 172.21.15.19:0/2077650566 shutdown_connections 2024-03-24T02:53:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.579+0000 7ff43922d700 1 --2- 172.21.15.19:0/2077650566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff434103d60 0x7ff43410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.579+0000 7ff43922d700 1 --2- 172.21.15.19:0/2077650566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4341021c0 0x7ff4341025e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.579+0000 7ff43922d700 1 --2- 172.21.15.19:0/2077650566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4341033c0 0x7ff434103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.579+0000 7ff43922d700 1 -- 172.21.15.19:0/2077650566 >> 172.21.15.19:0/2077650566 conn(0x7ff4340fd7a0 msgr2=0x7ff4340ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:53:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.579+0000 7ff43922d700 1 -- 172.21.15.19:0/2077650566 shutdown_connections 2024-03-24T02:53:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.579+0000 7ff43922d700 1 -- 172.21.15.19:0/2077650566 wait complete. 2024-03-24T02:53:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.580+0000 7ff43922d700 1 Processor -- start 2024-03-24T02:53:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.580+0000 7ff43922d700 1 -- start start 2024-03-24T02:53:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.580+0000 7ff43922d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4341021c0 0x7ff434198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:53:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.580+0000 7ff43922d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4341033c0 0x7ff434198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:53:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43922d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff434103d60 0x7ff43419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:53:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43922d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43410dda0 con 0x7ff4341021c0 2024-03-24T02:53:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43922d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43410dc20 con 0x7ff434103d60 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43922d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43410df20 con 0x7ff4341033c0 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff434103d60 0x7ff43419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43259c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4341033c0 0x7ff434198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff434103d60 0x7ff43419f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51886/0 (socket says 172.21.15.19:51886) 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43359e700 1 -- 172.21.15.19:0/1664450000 learned_addr learned my addr 172.21.15.19:0/1664450000 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff432d9d700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4341021c0 0x7ff434198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43259c700 1 -- 172.21.15.19:0/1664450000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff434103d60 msgr2=0x7ff43419f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43259c700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff434103d60 0x7ff43419f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43259c700 1 -- 172.21.15.19:0/1664450000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4341021c0 msgr2=0x7ff434198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43259c700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4341021c0 0x7ff434198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43259c700 1 -- 172.21.15.19:0/1664450000 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4341a0070 con 0x7ff4341033c0 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43359e700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff434103d60 0x7ff43419f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:53:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43259c700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4341033c0 0x7ff434198950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff42800d980 tx=0x7ff42800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:53:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff41bfff700 1 -- 172.21.15.19:0/1664450000 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff428017070 con 0x7ff4341033c0 2024-03-24T02:53:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff41bfff700 1 -- 172.21.15.19:0/1664450000 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff42800fd70 con 0x7ff4341033c0 2024-03-24T02:53:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff43922d700 1 -- 172.21.15.19:0/1664450000 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4341a0360 con 0x7ff4341033c0 2024-03-24T02:53:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.581+0000 7ff41bfff700 1 -- 172.21.15.19:0/1664450000 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42800e3b0 con 0x7ff4341033c0 2024-03-24T02:53:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.582+0000 7ff43922d700 1 -- 172.21.15.19:0/1664450000 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4341a08a0 con 0x7ff4341033c0 2024-03-24T02:53:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.582+0000 7ff43922d700 1 -- 172.21.15.19:0/1664450000 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff434068e50 con 0x7ff4341033c0 2024-03-24T02:53:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.584+0000 7ff41bfff700 1 -- 172.21.15.19:0/1664450000 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7ff42801a070 con 0x7ff4341033c0 2024-03-24T02:53:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.584+0000 7ff41bfff700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff41c0a2800 0x7ff41c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:53:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.584+0000 7ff41bfff700 1 -- 172.21.15.19:0/1664450000 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7ff4280cc7f0 con 0x7ff4341033c0 2024-03-24T02:53:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.584+0000 7ff432d9d700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff41c0a2800 0x7ff41c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:53:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.585+0000 7ff432d9d700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff41c0a2800 0x7ff41c0a4cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff42400b580 tx=0x7ff424005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:53:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.588+0000 7ff41bfff700 1 -- 172.21.15.19:0/1664450000 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff428010460 con 0x7ff4341033c0 2024-03-24T02:53:07.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:07.805+0000 7ff43922d700 1 -- 172.21.15.19:0/1664450000 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff434062690 con 0x7ff41c0a2800 2024-03-24T02:53:07.807 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:07 smithi019 ceph-mon[35944]: Detected new or changed devices on smithi017 2024-03-24T02:53:07.807 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:07 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi017 to 17093M 2024-03-24T02:53:07.807 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:07 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:07.807 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:07 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:07.807 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:07 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:07 smithi017 ceph-mon[31926]: Detected new or changed devices on smithi017 2024-03-24T02:53:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:07 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi017 to 17093M 2024-03-24T02:53:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:07 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:07 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:07 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:08.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:07 smithi188 ceph-mon[36206]: Detected new or changed devices on smithi017 2024-03-24T02:53:08.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:07 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi017 to 17093M 2024-03-24T02:53:08.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:07 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:08.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:07 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:08.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:07 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:08.722 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:08 smithi019 ceph-mon[35944]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:53:08.723 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:53:08.723 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:53:08.723 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:08.723 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:08 smithi019 ceph-mon[35944]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:08.723 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:08 smithi019 ceph-mon[35944]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T02:53:08.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:08 smithi017 ceph-mon[31926]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:53:08.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:53:08.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:53:08.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:08.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:08 smithi017 ceph-mon[31926]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:08.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:08 smithi017 ceph-mon[31926]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T02:53:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:08 smithi188 ceph-mon[36206]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:53:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:53:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:53:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:08 smithi188 ceph-mon[36206]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:08 smithi188 ceph-mon[36206]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T02:53:11.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:10 smithi019 ceph-mon[35944]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:11.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:10 smithi019 ceph-mon[35944]: from='client.? 172.21.15.19:0/3716421054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83235b08-6883-4443-8f53-7cd7f3db924f"}]: dispatch 2024-03-24T02:53:11.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:10 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83235b08-6883-4443-8f53-7cd7f3db924f"}]: dispatch 2024-03-24T02:53:11.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:10 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83235b08-6883-4443-8f53-7cd7f3db924f"}]': finished 2024-03-24T02:53:11.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:10 smithi019 ceph-mon[35944]: osdmap e10: 2 total, 1 up, 2 in 2024-03-24T02:53:11.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:10 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:10 smithi017 ceph-mon[31926]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:10 smithi017 ceph-mon[31926]: from='client.? 172.21.15.19:0/3716421054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83235b08-6883-4443-8f53-7cd7f3db924f"}]: dispatch 2024-03-24T02:53:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:10 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83235b08-6883-4443-8f53-7cd7f3db924f"}]: dispatch 2024-03-24T02:53:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:10 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83235b08-6883-4443-8f53-7cd7f3db924f"}]': finished 2024-03-24T02:53:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:10 smithi017 ceph-mon[31926]: osdmap e10: 2 total, 1 up, 2 in 2024-03-24T02:53:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:10 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:11.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:10 smithi188 ceph-mon[36206]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:11.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:10 smithi188 ceph-mon[36206]: from='client.? 172.21.15.19:0/3716421054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83235b08-6883-4443-8f53-7cd7f3db924f"}]: dispatch 2024-03-24T02:53:11.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:10 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83235b08-6883-4443-8f53-7cd7f3db924f"}]: dispatch 2024-03-24T02:53:11.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:10 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83235b08-6883-4443-8f53-7cd7f3db924f"}]': finished 2024-03-24T02:53:11.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:10 smithi188 ceph-mon[36206]: osdmap e10: 2 total, 1 up, 2 in 2024-03-24T02:53:11.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:10 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:12.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:11 smithi017 ceph-mon[31926]: from='client.? 172.21.15.19:0/3186191569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:53:12.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:11 smithi188 ceph-mon[36206]: from='client.? 172.21.15.19:0/3186191569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:53:12.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:11 smithi019 ceph-mon[35944]: from='client.? 172.21.15.19:0/3186191569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:53:13.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:12 smithi017 ceph-mon[31926]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:13.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:12 smithi188 ceph-mon[36206]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:13.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:12 smithi019 ceph-mon[35944]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:13 smithi017 ceph-mon[31926]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:14.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:13 smithi188 ceph-mon[36206]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:14.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:13 smithi019 ceph-mon[35944]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:16.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:15 smithi017 ceph-mon[31926]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:16.290 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:15 smithi019 ceph-mon[35944]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:16.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:15 smithi188 ceph-mon[36206]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:17.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T02:53:17.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:18.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T02:53:18.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:18.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T02:53:18.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:18.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:18 smithi019 ceph-mon[35944]: Deploying daemon osd.1 on smithi019 2024-03-24T02:53:18.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:18 smithi019 ceph-mon[35944]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:18.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:18 smithi017 ceph-mon[31926]: Deploying daemon osd.1 on smithi019 2024-03-24T02:53:18.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:18 smithi017 ceph-mon[31926]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:19.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:18 smithi188 ceph-mon[36206]: Deploying daemon osd.1 on smithi019 2024-03-24T02:53:19.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:18 smithi188 ceph-mon[36206]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:19 smithi017 ceph-mon[31926]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:20.285 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:19 smithi019 ceph-mon[35944]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:20.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:19 smithi188 ceph-mon[36206]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:22.047 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:21 smithi019 ceph-mon[35944]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:21 smithi017 ceph-mon[31926]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:22.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:21 smithi188 ceph-mon[36206]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:23 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:23 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:23 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:24.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:23 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:24.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:23 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:24.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:23 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:24.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:23 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:24.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:23 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:24.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:23 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:25.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:24 smithi019 ceph-mon[35944]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:25.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:24 smithi017 ceph-mon[31926]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:25.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:24 smithi188 ceph-mon[36206]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:26.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.018+0000 7ff41bfff700 1 -- 172.21.15.19:0/1664450000 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff434062690 con 0x7ff41c0a2800 2024-03-24T02:53:26.019 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 1 on host 'smithi019' 2024-03-24T02:53:26.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.022+0000 7ff43922d700 1 -- 172.21.15.19:0/1664450000 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff41c0a2800 msgr2=0x7ff41c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:26.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.022+0000 7ff43922d700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff41c0a2800 0x7ff41c0a4cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff42400b580 tx=0x7ff424005fb0 comp rx=0 tx=0).stop 2024-03-24T02:53:26.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.022+0000 7ff43922d700 1 -- 172.21.15.19:0/1664450000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4341033c0 msgr2=0x7ff434198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:26.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.022+0000 7ff43922d700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4341033c0 0x7ff434198950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff42800d980 tx=0x7ff42800dc90 comp rx=0 tx=0).stop 2024-03-24T02:53:26.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.022+0000 7ff43922d700 1 -- 172.21.15.19:0/1664450000 shutdown_connections 2024-03-24T02:53:26.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.022+0000 7ff43922d700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff41c0a2800 0x7ff41c0a4cc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:26.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.022+0000 7ff43922d700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff434103d60 0x7ff43419f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:26.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.022+0000 7ff43922d700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4341021c0 0x7ff434198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:26.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.022+0000 7ff43922d700 1 --2- 172.21.15.19:0/1664450000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4341033c0 0x7ff434198950 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:26.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.022+0000 7ff43922d700 1 -- 172.21.15.19:0/1664450000 >> 172.21.15.19:0/1664450000 conn(0x7ff4340fd7a0 msgr2=0x7ff434106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:53:26.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.022+0000 7ff43922d700 1 -- 172.21.15.19:0/1664450000 shutdown_connections 2024-03-24T02:53:26.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:53:26.022+0000 7ff43922d700 1 -- 172.21.15.19:0/1664450000 wait complete. 2024-03-24T02:53:26.657 DEBUG:teuthology.orchestra.run.smithi019:osd.1> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.1.service 2024-03-24T02:53:26.659 INFO:tasks.cephadm:Deploying osd.2 on smithi188 with /dev/vg_nvme/lv_4... 2024-03-24T02:53:26.659 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-24T02:53:26.890 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 02:53:26 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1[40541]: 2024-03-24T02:53:26.797+0000 7f13d08107c0 -1 osd.1 0 log_to_monitors true 2024-03-24T02:53:26.906 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T02:53:27.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:26 smithi188 ceph-mon[36206]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:27.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:26 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:27.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:26 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:27.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:26 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:27.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:26 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:27.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:26 smithi188 ceph-mon[36206]: from='osd.1 [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T02:53:27.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:26 smithi188 ceph-mon[36206]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T02:53:27.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:26 smithi017 ceph-mon[31926]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:27.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:26 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:27.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:26 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:27.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:26 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:27.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:26 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:27.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:26 smithi017 ceph-mon[31926]: from='osd.1 [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T02:53:27.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:26 smithi017 ceph-mon[31926]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T02:53:27.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:26 smithi019 ceph-mon[35944]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:27.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:26 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:27.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:26 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:27.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:26 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:27.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:26 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:27.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:26 smithi019 ceph-mon[35944]: from='osd.1 [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T02:53:27.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:26 smithi019 ceph-mon[35944]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T02:53:28.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:28 smithi019 ceph-mon[35944]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T02:53:28.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:28 smithi019 ceph-mon[35944]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T02:53:28.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:28 smithi019 ceph-mon[35944]: from='osd.1 [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:53:28.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:28 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:28.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:28 smithi019 ceph-mon[35944]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:53:28.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:28 smithi019 ceph-mon[35944]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:28.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:28 smithi188 ceph-mon[36206]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T02:53:28.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:28 smithi188 ceph-mon[36206]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T02:53:28.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:28 smithi188 ceph-mon[36206]: from='osd.1 [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:53:28.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:28 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:28.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:28 smithi188 ceph-mon[36206]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:53:28.213 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:28 smithi188 ceph-mon[36206]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:28.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:28 smithi017 ceph-mon[31926]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T02:53:28.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:28 smithi017 ceph-mon[31926]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T02:53:28.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:28 smithi017 ceph-mon[31926]: from='osd.1 [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:53:28.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:28 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:28.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:28 smithi017 ceph-mon[31926]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:53:28.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:28 smithi017 ceph-mon[31926]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:28.788 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-24T02:53:28.788 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-24T02:53:28.788 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-03-24T02:53:28.788 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-03-24T02:53:28.788 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129093 s, 812 MB/s 2024-03-24T02:53:28.788 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-03-24T02:53:29.088 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 02:53:28 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1[40541]: 2024-03-24T02:53:28.819+0000 7f13c6676700 -1 osd.1 0 waiting for initial osdmap 2024-03-24T02:53:29.088 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 02:53:28 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1[40541]: 2024-03-24T02:53:28.824+0000 7f13bdc60700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T02:53:29.309 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi188:vg_nvme/lv_4 2024-03-24T02:53:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:29 smithi017 ceph-mon[31926]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-24T02:53:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:29 smithi017 ceph-mon[31926]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T02:53:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:29 smithi017 ceph-mon[31926]: from='osd.1 ' entity='osd.1' 2024-03-24T02:53:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:29 smithi188 ceph-mon[36206]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-24T02:53:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:29 smithi188 ceph-mon[36206]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T02:53:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:29 smithi188 ceph-mon[36206]: from='osd.1 ' entity='osd.1' 2024-03-24T02:53:29.550 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T02:53:29.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:29 smithi019 ceph-mon[35944]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-24T02:53:29.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:29 smithi019 ceph-mon[35944]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T02:53:29.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:29.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:29.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:29 smithi019 ceph-mon[35944]: from='osd.1 ' entity='osd.1' 2024-03-24T02:53:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: purged_snaps scrub starts 2024-03-24T02:53:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: purged_snaps scrub ok 2024-03-24T02:53:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: Detected new or changed devices on smithi019 2024-03-24T02:53:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:53:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi019 to 17084M 2024-03-24T02:53:30.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:30.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:30.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: osd.1 [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517] boot 2024-03-24T02:53:30.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: osdmap e13: 2 total, 2 up, 2 in 2024-03-24T02:53:30.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:30.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:30.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:30.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:30.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:30.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.356 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: purged_snaps scrub starts 2024-03-24T02:53:30.356 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: purged_snaps scrub ok 2024-03-24T02:53:30.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:30.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: Detected new or changed devices on smithi019 2024-03-24T02:53:30.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:53:30.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi019 to 17084M 2024-03-24T02:53:30.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:30.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:30.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: osd.1 [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517] boot 2024-03-24T02:53:30.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: osdmap e13: 2 total, 2 up, 2 in 2024-03-24T02:53:30.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:30.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:30.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:30.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:30.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:30.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: purged_snaps scrub starts 2024-03-24T02:53:30.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: purged_snaps scrub ok 2024-03-24T02:53:30.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: Detected new or changed devices on smithi019 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi019 to 17084M 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: osd.1 [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517] boot 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: osdmap e13: 2 total, 2 up, 2 in 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:30.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:30.572 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:31.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.154+0000 7f5a07b49700 1 -- 172.21.15.188:0/3071373495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a00100c50 msgr2=0x7f5a0010a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:31.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.154+0000 7f5a07b49700 1 --2- 172.21.15.188:0/3071373495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a00100c50 0x7f5a0010a820 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f59f800b0a0 tx=0x7f59f800b3b0 comp rx=0 tx=0).stop 2024-03-24T02:53:31.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.154+0000 7f5a07b49700 1 -- 172.21.15.188:0/3071373495 shutdown_connections 2024-03-24T02:53:31.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.154+0000 7f5a07b49700 1 --2- 172.21.15.188:0/3071373495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a00100c50 0x7f5a0010a820 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:31.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.154+0000 7f5a07b49700 1 --2- 172.21.15.188:0/3071373495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a00103830 0x7f5a00103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.154+0000 7f5a07b49700 1 --2- 172.21.15.188:0/3071373495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a001002b0 0x7f5a00100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.154+0000 7f5a07b49700 1 -- 172.21.15.188:0/3071373495 >> 172.21.15.188:0/3071373495 conn(0x7f5a000fbe30 msgr2=0x7f5a000fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.154+0000 7f5a07b49700 1 -- 172.21.15.188:0/3071373495 shutdown_connections 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.154+0000 7f5a07b49700 1 -- 172.21.15.188:0/3071373495 wait complete. 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.155+0000 7f5a07b49700 1 Processor -- start 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.155+0000 7f5a07b49700 1 -- start start 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.155+0000 7f5a07b49700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a001002b0 0x7f5a00198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.155+0000 7f5a07b49700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a00100c50 0x7f5a00198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.155+0000 7f5a07b49700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a00103830 0x7f5a0019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.155+0000 7f5a07b49700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0010dde0 con 0x7f5a00103830 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.155+0000 7f5a07b49700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0010dc60 con 0x7f5a00100c50 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.155+0000 7f5a07b49700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0010df60 con 0x7f5a001002b0 2024-03-24T02:53:31.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.155+0000 7f5a058e5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a001002b0 0x7f5a00198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.155+0000 7f5a058e5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a001002b0 0x7f5a00198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:57270/0 (socket says 172.21.15.188:57270) 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.155+0000 7f5a058e5700 1 -- 172.21.15.188:0/4207138936 learned_addr learned my addr 172.21.15.188:0/4207138936 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.155+0000 7f5a060e6700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a00103830 0x7f5a0019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.156+0000 7f5a058e5700 1 -- 172.21.15.188:0/4207138936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a00100c50 msgr2=0x7f5a00198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.156+0000 7f5a058e5700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a00100c50 0x7f5a00198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.156+0000 7f5a058e5700 1 -- 172.21.15.188:0/4207138936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a00103830 msgr2=0x7f5a0019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.156+0000 7f5a058e5700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a00103830 0x7f5a0019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.156+0000 7f5a058e5700 1 -- 172.21.15.188:0/4207138936 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0019ff60 con 0x7f5a001002b0 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.156+0000 7f5a060e6700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a00103830 0x7f5a0019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.156+0000 7f5a058e5700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a001002b0 0x7f5a00198300 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59f000c9a0 tx=0x7f59f000cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.156+0000 7f59f6ffd700 1 -- 172.21.15.188:0/4207138936 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0004220 con 0x7f5a001002b0 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.156+0000 7f59f6ffd700 1 -- 172.21.15.188:0/4207138936 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59f0007d40 con 0x7f5a001002b0 2024-03-24T02:53:31.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.156+0000 7f5a07b49700 1 -- 172.21.15.188:0/4207138936 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a001a0250 con 0x7f5a001002b0 2024-03-24T02:53:31.159 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.156+0000 7f59f6ffd700 1 -- 172.21.15.188:0/4207138936 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f000f450 con 0x7f5a001002b0 2024-03-24T02:53:31.159 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.156+0000 7f5a07b49700 1 -- 172.21.15.188:0/4207138936 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a001a0790 con 0x7f5a001002b0 2024-03-24T02:53:31.159 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.157+0000 7f5a07b49700 1 -- 172.21.15.188:0/4207138936 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a00068e50 con 0x7f5a001002b0 2024-03-24T02:53:31.160 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.158+0000 7f59f6ffd700 1 -- 172.21.15.188:0/4207138936 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f59f0007860 con 0x7f5a001002b0 2024-03-24T02:53:31.160 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.159+0000 7f59f6ffd700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f59ec0a2800 0x7f59ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:53:31.160 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.159+0000 7f59f6ffd700 1 -- 172.21.15.188:0/4207138936 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f59f00cd1b0 con 0x7f5a001002b0 2024-03-24T02:53:31.160 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.159+0000 7f5a050e4700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f59ec0a2800 0x7f59ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:53:31.160 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.160+0000 7f5a050e4700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f59ec0a2800 0x7f59ec0a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59fc005950 tx=0x7f59fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:53:31.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.164+0000 7f59f6ffd700 1 -- 172.21.15.188:0/4207138936 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f59f0014030 con 0x7f5a001002b0 2024-03-24T02:53:31.366 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:31.366+0000 7f5a07b49700 1 -- 172.21.15.188:0/4207138936 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a000625d0 con 0x7f59ec0a2800 2024-03-24T02:53:32.081 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:31 smithi188 ceph-mon[36206]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T02:53:32.082 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:53:32.082 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:53:32.082 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:32.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:31 smithi017 ceph-mon[31926]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T02:53:32.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:53:32.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:53:32.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:31 smithi019 ceph-mon[35944]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T02:53:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:53:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:53:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:32.900 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:32 smithi188 ceph-mon[36206]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:53:32.901 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:32 smithi188 ceph-mon[36206]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:33.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:32 smithi017 ceph-mon[31926]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:53:33.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:32 smithi017 ceph-mon[31926]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:33.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:32 smithi019 ceph-mon[35944]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:53:33.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:32 smithi019 ceph-mon[35944]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:33 smithi017 ceph-mon[31926]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:34.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:33 smithi188 ceph-mon[36206]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:34.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:33 smithi019 ceph-mon[35944]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:34 smithi017 ceph-mon[31926]: from='client.? 172.21.15.188:0/3524257452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6"}]: dispatch 2024-03-24T02:53:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:34 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6"}]: dispatch 2024-03-24T02:53:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:34 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6"}]': finished 2024-03-24T02:53:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:34 smithi017 ceph-mon[31926]: osdmap e15: 3 total, 2 up, 3 in 2024-03-24T02:53:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:34 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:34 smithi017 ceph-mon[31926]: from='client.? 172.21.15.188:0/2728934624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:53:35.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:34 smithi188 ceph-mon[36206]: from='client.? 172.21.15.188:0/3524257452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6"}]: dispatch 2024-03-24T02:53:35.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:34 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6"}]: dispatch 2024-03-24T02:53:35.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:34 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6"}]': finished 2024-03-24T02:53:35.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:34 smithi188 ceph-mon[36206]: osdmap e15: 3 total, 2 up, 3 in 2024-03-24T02:53:35.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:34 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:35.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:34 smithi188 ceph-mon[36206]: from='client.? 172.21.15.188:0/2728934624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:53:35.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:34 smithi019 ceph-mon[35944]: from='client.? 172.21.15.188:0/3524257452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6"}]: dispatch 2024-03-24T02:53:35.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:34 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6"}]: dispatch 2024-03-24T02:53:35.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:34 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6"}]': finished 2024-03-24T02:53:35.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:34 smithi019 ceph-mon[35944]: osdmap e15: 3 total, 2 up, 3 in 2024-03-24T02:53:35.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:34 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:35.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:34 smithi019 ceph-mon[35944]: from='client.? 172.21.15.188:0/2728934624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:53:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:35 smithi017 ceph-mon[31926]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:36.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:35 smithi188 ceph-mon[36206]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:35 smithi019 ceph-mon[35944]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:38.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:37 smithi017 ceph-mon[31926]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:38.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:37 smithi188 ceph-mon[36206]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:37 smithi019 ceph-mon[35944]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:40.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:39 smithi017 ceph-mon[31926]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:40.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:39 smithi188 ceph-mon[36206]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:39 smithi019 ceph-mon[35944]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:41.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T02:53:41.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T02:53:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T02:53:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:42.116 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:42 smithi188 ceph-mon[36206]: Deploying daemon osd.2 on smithi188 2024-03-24T02:53:42.116 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:42 smithi188 ceph-mon[36206]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:42 smithi019 ceph-mon[35944]: Deploying daemon osd.2 on smithi188 2024-03-24T02:53:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:42 smithi019 ceph-mon[35944]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:42.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:42 smithi017 ceph-mon[31926]: Deploying daemon osd.2 on smithi188 2024-03-24T02:53:42.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:42 smithi017 ceph-mon[31926]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:44.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:43 smithi188 ceph-mon[36206]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:44.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:43 smithi017 ceph-mon[31926]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:43 smithi019 ceph-mon[35944]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:46.191 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:45 smithi188 ceph-mon[36206]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:46.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:45 smithi017 ceph-mon[31926]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:46.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:45 smithi019 ceph-mon[35944]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:47.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:47 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:47.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:47 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:47.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:47 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:47.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:47 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:47.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:47 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:47.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:47 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:47.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:47 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:47.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:47 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:47.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:47 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:48.354 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:48 smithi188 ceph-mon[36206]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:48.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:48 smithi019 ceph-mon[35944]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:48.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:48 smithi017 ceph-mon[31926]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:49.484 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 2 on host 'smithi188' 2024-03-24T02:53:49.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.484+0000 7f59f6ffd700 1 -- 172.21.15.188:0/4207138936 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5a000625d0 con 0x7f59ec0a2800 2024-03-24T02:53:49.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.486+0000 7f5a07b49700 1 -- 172.21.15.188:0/4207138936 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f59ec0a2800 msgr2=0x7f59ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:49.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.486+0000 7f5a07b49700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f59ec0a2800 0x7f59ec0a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59fc005950 tx=0x7f59fc0058e0 comp rx=0 tx=0).stop 2024-03-24T02:53:49.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.486+0000 7f5a07b49700 1 -- 172.21.15.188:0/4207138936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a001002b0 msgr2=0x7f5a00198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:49.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.486+0000 7f5a07b49700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a001002b0 0x7f5a00198300 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59f000c9a0 tx=0x7f59f000cd60 comp rx=0 tx=0).stop 2024-03-24T02:53:49.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.486+0000 7f5a07b49700 1 -- 172.21.15.188:0/4207138936 shutdown_connections 2024-03-24T02:53:49.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.486+0000 7f5a07b49700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f59ec0a2800 0x7f59ec0a4cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:49.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.486+0000 7f5a07b49700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a00103830 0x7f5a0019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:49.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.486+0000 7f5a07b49700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a001002b0 0x7f5a00198300 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:49.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.486+0000 7f5a07b49700 1 --2- 172.21.15.188:0/4207138936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a00100c50 0x7f5a00198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:49.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.486+0000 7f5a07b49700 1 -- 172.21.15.188:0/4207138936 >> 172.21.15.188:0/4207138936 conn(0x7f5a000fbe30 msgr2=0x7f5a00106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:53:49.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.486+0000 7f5a07b49700 1 -- 172.21.15.188:0/4207138936 shutdown_connections 2024-03-24T02:53:49.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:53:49.486+0000 7f5a07b49700 1 -- 172.21.15.188:0/4207138936 wait complete. 2024-03-24T02:53:50.097 DEBUG:teuthology.orchestra.run.smithi188:osd.2> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.2.service 2024-03-24T02:53:50.100 INFO:tasks.cephadm:Deploying osd.3 on smithi017 with /dev/vg_nvme/lv_3... 2024-03-24T02:53:50.100 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-24T02:53:50.353 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:53:50.407 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:50 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.408 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:50 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.408 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:50 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:50.408 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:50 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:50.408 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:50 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.408 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:50 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.408 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:50 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.408 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:50 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:50.408 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:50 smithi188 ceph-mon[36206]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:50 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:50 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:50 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:50 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:50 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:50 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:50 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:50 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:50 smithi017 ceph-mon[31926]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:50.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:50 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:50 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:50 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:50.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:50 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:50.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:50 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:50 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:50 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:50.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:50 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:53:50.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:50 smithi019 ceph-mon[35944]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:51.412 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:51 smithi188 ceph-mon[36206]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T02:53:51.412 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:51 smithi017 ceph-mon[31926]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T02:53:51.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:51 smithi019 ceph-mon[35944]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T02:53:52.236 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:52 smithi017 ceph-mon[31926]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T02:53:52.236 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:52 smithi017 ceph-mon[31926]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T02:53:52.236 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:52 smithi017 ceph-mon[31926]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:53:52.236 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:52 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:52.236 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:52 smithi017 ceph-mon[31926]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:52.237 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-24T02:53:52.237 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-24T02:53:52.237 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2024-03-24T02:53:52.237 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2024-03-24T02:53:52.237 INFO:teuthology.orchestra.run.smithi017.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00840259 s, 1.2 GB/s 2024-03-24T02:53:52.237 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2024-03-24T02:53:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:52 smithi019 ceph-mon[35944]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T02:53:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:52 smithi019 ceph-mon[35944]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T02:53:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:52 smithi019 ceph-mon[35944]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:53:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:52 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:52 smithi019 ceph-mon[35944]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:52.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:52 smithi188 ceph-mon[36206]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T02:53:52.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:52 smithi188 ceph-mon[36206]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T02:53:52.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:52 smithi188 ceph-mon[36206]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:53:52.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:52 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:52.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:52 smithi188 ceph-mon[36206]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:52.888 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi017:vg_nvme/lv_3 2024-03-24T02:53:52.935 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 02:53:52 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2[41347]: 2024-03-24T02:53:52.823+0000 7fdddc337700 -1 osd.2 0 waiting for initial osdmap 2024-03-24T02:53:52.936 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 02:53:52 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2[41347]: 2024-03-24T02:53:52.829+0000 7fddd4122700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T02:53:53.137 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:53:53.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: purged_snaps scrub starts 2024-03-24T02:53:53.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: purged_snaps scrub ok 2024-03-24T02:53:53.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-03-24T02:53:53.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T02:53:53.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:53.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:53.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' 2024-03-24T02:53:53.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:53.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:53.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:53:53.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:53.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:53.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:53.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:53.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: purged_snaps scrub starts 2024-03-24T02:53:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: purged_snaps scrub ok 2024-03-24T02:53:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-03-24T02:53:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T02:53:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' 2024-03-24T02:53:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:53:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:53.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: purged_snaps scrub starts 2024-03-24T02:53:53.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: purged_snaps scrub ok 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: from='osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482]' entity='osd.2' 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:54.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.793+0000 7f8c9921c700 1 -- 172.21.15.17:0/3136104027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c94102670 msgr2=0x7f8c94102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:54.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.793+0000 7f8c9921c700 1 --2- 172.21.15.17:0/3136104027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c94102670 0x7f8c94102a90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8c840099c0 tx=0x7f8c84009cd0 comp rx=0 tx=0).stop 2024-03-24T02:53:54.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.793+0000 7f8c9921c700 1 -- 172.21.15.17:0/3136104027 shutdown_connections 2024-03-24T02:53:54.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.793+0000 7f8c9921c700 1 --2- 172.21.15.17:0/3136104027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c94101d50 0x7f8c94109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.793+0000 7f8c9921c700 1 --2- 172.21.15.17:0/3136104027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c94102670 0x7f8c94102a90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.793+0000 7f8c9921c700 1 --2- 172.21.15.17:0/3136104027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c941013b0 0x7f8c94101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.793+0000 7f8c9921c700 1 -- 172.21.15.17:0/3136104027 >> 172.21.15.17:0/3136104027 conn(0x7f8c940fbde0 msgr2=0x7f8c940fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:53:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.793+0000 7f8c9921c700 1 -- 172.21.15.17:0/3136104027 shutdown_connections 2024-03-24T02:53:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.793+0000 7f8c9921c700 1 -- 172.21.15.17:0/3136104027 wait complete. 2024-03-24T02:53:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.793+0000 7f8c9921c700 1 Processor -- start 2024-03-24T02:53:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c9921c700 1 -- start start 2024-03-24T02:53:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c9921c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c941013b0 0x7f8c94105100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:53:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c9921c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c94101d50 0x7f8c941037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:53:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c9921c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c94102670 0x7f8c94103ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:53:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c9921c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9410ccd0 con 0x7f8c941013b0 2024-03-24T02:53:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c9921c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9410cb50 con 0x7f8c94101d50 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c9921c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9410ce50 con 0x7f8c94102670 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c92d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c941013b0 0x7f8c94105100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c92d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c941013b0 0x7f8c94105100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57962/0 (socket says 172.21.15.17:57962) 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c92d9d700 1 -- 172.21.15.17:0/1007682930 learned_addr learned my addr 172.21.15.17:0/1007682930 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c9259c700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c94101d50 0x7f8c941037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c9359e700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c94102670 0x7f8c94103ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c92d9d700 1 -- 172.21.15.17:0/1007682930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c94102670 msgr2=0x7f8c94103ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c92d9d700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c94102670 0x7f8c94103ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c92d9d700 1 -- 172.21.15.17:0/1007682930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c94101d50 msgr2=0x7f8c941037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c92d9d700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c94101d50 0x7f8c941037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c92d9d700 1 -- 172.21.15.17:0/1007682930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c941045b0 con 0x7f8c941013b0 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c9259c700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c94101d50 0x7f8c941037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.794+0000 7f8c92d9d700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c941013b0 0x7f8c94105100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400c010 tx=0x7f8c84009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:53:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.795+0000 7f8c7bfff700 1 -- 172.21.15.17:0/1007682930 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84027070 con 0x7f8c941013b0 2024-03-24T02:53:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.795+0000 7f8c7bfff700 1 -- 172.21.15.17:0/1007682930 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c84004150 con 0x7f8c941013b0 2024-03-24T02:53:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.795+0000 7f8c9921c700 1 -- 172.21.15.17:0/1007682930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c941a2550 con 0x7f8c941013b0 2024-03-24T02:53:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.795+0000 7f8c7bfff700 1 -- 172.21.15.17:0/1007682930 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84022ce0 con 0x7f8c941013b0 2024-03-24T02:53:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.795+0000 7f8c9921c700 1 -- 172.21.15.17:0/1007682930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c941a2ab0 con 0x7f8c941013b0 2024-03-24T02:53:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.796+0000 7f8c9921c700 1 -- 172.21.15.17:0/1007682930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c94068e50 con 0x7f8c941013b0 2024-03-24T02:53:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.797+0000 7f8c7bfff700 1 -- 172.21.15.17:0/1007682930 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f8c8402a070 con 0x7f8c941013b0 2024-03-24T02:53:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.798+0000 7f8c7bfff700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8c7c0a2880 0x7f8c7c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:53:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.798+0000 7f8c9259c700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8c7c0a2880 0x7f8c7c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:53:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.798+0000 7f8c7bfff700 1 -- 172.21.15.17:0/1007682930 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f8c840ceaa0 con 0x7f8c941013b0 2024-03-24T02:53:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.798+0000 7f8c9259c700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8c7c0a2880 0x7f8c7c0a4d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8c880104e0 tx=0x7f8c8800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:53:54.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:54.802+0000 7f8c7bfff700 1 -- 172.21.15.17:0/1007682930 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8c84031020 con 0x7f8c941013b0 2024-03-24T02:53:55.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:55.022+0000 7f8c9921c700 1 -- 172.21.15.17:0/1007682930 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c941a2d10 con 0x7f8c7c0a2880 2024-03-24T02:53:55.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:54 smithi017 ceph-mon[31926]: Detected new or changed devices on smithi188 2024-03-24T02:53:55.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:54 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi188 to 17080M 2024-03-24T02:53:55.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:54 smithi017 ceph-mon[31926]: osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482] boot 2024-03-24T02:53:55.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:54 smithi017 ceph-mon[31926]: osdmap e18: 3 total, 3 up, 3 in 2024-03-24T02:53:55.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:54 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:55.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:54 smithi017 ceph-mon[31926]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:55.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:54 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-24T02:53:55.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:54 smithi188 ceph-mon[36206]: Detected new or changed devices on smithi188 2024-03-24T02:53:55.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:54 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi188 to 17080M 2024-03-24T02:53:55.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:54 smithi188 ceph-mon[36206]: osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482] boot 2024-03-24T02:53:55.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:54 smithi188 ceph-mon[36206]: osdmap e18: 3 total, 3 up, 3 in 2024-03-24T02:53:55.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:54 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:55.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:54 smithi188 ceph-mon[36206]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:55.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:54 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-24T02:53:55.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:54 smithi019 ceph-mon[35944]: Detected new or changed devices on smithi188 2024-03-24T02:53:55.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:54 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi188 to 17080M 2024-03-24T02:53:55.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:54 smithi019 ceph-mon[35944]: osd.2 [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482] boot 2024-03-24T02:53:55.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:54 smithi019 ceph-mon[35944]: osdmap e18: 3 total, 3 up, 3 in 2024-03-24T02:53:55.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:54 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T02:53:55.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:54 smithi019 ceph-mon[35944]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-24T02:53:55.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:54 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-24T02:53:55.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:55 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-24T02:53:55.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:55 smithi017 ceph-mon[31926]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T02:53:55.890 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:55 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T02:53:55.890 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:55 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:53:55.890 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:55 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:53:55.890 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:55 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:56.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:55 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-24T02:53:56.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:55 smithi188 ceph-mon[36206]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T02:53:56.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:55 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T02:53:56.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:55 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:53:56.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:55 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:53:56.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:55 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:55 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-24T02:53:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:55 smithi019 ceph-mon[35944]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T02:53:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:55 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T02:53:56.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:55 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:53:56.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:55 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:53:56.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:55 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:53:56.744 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 02:53:56 smithi017 sudo[50962]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T02:53:56.744 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 02:53:56 smithi017 sudo[50962]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T02:53:56.745 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 02:53:56 smithi017 sudo[50962]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T02:53:56.745 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 02:53:56 smithi017 sudo[50962]: pam_unix(sudo:session): session closed for user root 2024-03-24T02:53:56.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:53:56.851+0000 7f8c7bfff700 1 -- 172.21.15.17:0/1007682930 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8c84099290 con 0x7f8c941013b0 2024-03-24T02:53:57.096 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:56 smithi017 ceph-mon[31926]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:53:57.096 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:56 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T02:53:57.096 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:56 smithi017 ceph-mon[31926]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T02:53:57.096 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:56 smithi017 ceph-mon[31926]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:53:57.097 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:56 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:57.097 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 02:53:56 smithi017 sudo[50986]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T02:53:57.097 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 02:53:56 smithi017 sudo[50986]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T02:53:57.097 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 02:53:56 smithi017 sudo[50986]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T02:53:57.097 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 02:53:56 smithi017 sudo[50986]: pam_unix(sudo:session): session closed for user root 2024-03-24T02:53:57.218 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:56 smithi019 ceph-mon[35944]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:53:57.219 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:56 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T02:53:57.219 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:56 smithi019 ceph-mon[35944]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T02:53:57.219 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:56 smithi019 ceph-mon[35944]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:53:57.219 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:56 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:57.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:56 smithi188 ceph-mon[36206]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:53:57.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:56 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T02:53:57.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:56 smithi188 ceph-mon[36206]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T02:53:57.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:56 smithi188 ceph-mon[36206]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:53:57.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:56 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:57.570 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 02:53:57 smithi019 sudo[44258]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T02:53:57.570 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 02:53:57 smithi019 sudo[44258]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T02:53:57.570 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 02:53:57 smithi019 sudo[44258]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T02:53:57.570 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 02:53:57 smithi019 sudo[44258]: pam_unix(sudo:session): session closed for user root 2024-03-24T02:53:58.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:57 smithi019 ceph-mon[35944]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T02:53:58.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:57 smithi019 ceph-mon[35944]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-24T02:53:58.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:57 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:58.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:57 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4181334542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ba93700-990f-4c99-9d48-1adf99322cf8"}]: dispatch 2024-03-24T02:53:58.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:57 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4181334542' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ba93700-990f-4c99-9d48-1adf99322cf8"}]': finished 2024-03-24T02:53:58.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:57 smithi019 ceph-mon[35944]: osdmap e22: 4 total, 3 up, 4 in 2024-03-24T02:53:58.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:57 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:53:58.071 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 02:53:57 smithi019 sudo[44261]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T02:53:58.071 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 02:53:57 smithi019 sudo[44261]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T02:53:58.072 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 02:53:57 smithi019 sudo[44261]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T02:53:58.072 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 02:53:57 smithi019 sudo[44261]: pam_unix(sudo:session): session closed for user root 2024-03-24T02:53:58.195 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 02:53:57 smithi188 sudo[44983]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T02:53:58.195 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 02:53:57 smithi188 sudo[44983]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T02:53:58.195 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 02:53:57 smithi188 sudo[44983]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T02:53:58.195 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 02:53:57 smithi188 sudo[44983]: pam_unix(sudo:session): session closed for user root 2024-03-24T02:53:58.196 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:57 smithi188 ceph-mon[36206]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T02:53:58.196 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:57 smithi188 ceph-mon[36206]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-24T02:53:58.196 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:57 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:58.196 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:57 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4181334542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ba93700-990f-4c99-9d48-1adf99322cf8"}]: dispatch 2024-03-24T02:53:58.196 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:57 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4181334542' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ba93700-990f-4c99-9d48-1adf99322cf8"}]': finished 2024-03-24T02:53:58.196 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:57 smithi188 ceph-mon[36206]: osdmap e22: 4 total, 3 up, 4 in 2024-03-24T02:53:58.196 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:57 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:53:58.196 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:57 smithi017 ceph-mon[31926]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T02:53:58.196 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:57 smithi017 ceph-mon[31926]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-24T02:53:58.197 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:57 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:53:58.197 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:57 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4181334542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ba93700-990f-4c99-9d48-1adf99322cf8"}]: dispatch 2024-03-24T02:53:58.197 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:57 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4181334542' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ba93700-990f-4c99-9d48-1adf99322cf8"}]': finished 2024-03-24T02:53:58.197 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:57 smithi017 ceph-mon[31926]: osdmap e22: 4 total, 3 up, 4 in 2024-03-24T02:53:58.197 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:57 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:53:58.547 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 02:53:58 smithi188 sudo[44986]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T02:53:58.548 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 02:53:58 smithi188 sudo[44986]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T02:53:58.548 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 02:53:58 smithi188 sudo[44986]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T02:53:58.548 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 02:53:58 smithi188 sudo[44986]: pam_unix(sudo:session): session closed for user root 2024-03-24T02:53:58.741 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:58 smithi017 sudo[51074]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T02:53:58.742 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:58 smithi017 sudo[51074]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T02:53:58.742 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:58 smithi017 sudo[51074]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T02:53:58.742 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:58 smithi017 sudo[51074]: pam_unix(sudo:session): session closed for user root 2024-03-24T02:53:59.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:58 smithi017 sudo[51078]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T02:53:59.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:58 smithi017 sudo[51078]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T02:53:59.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:58 smithi017 sudo[51078]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T02:53:59.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:53:58 smithi017 sudo[51078]: pam_unix(sudo:session): session closed for user root 2024-03-24T02:53:59.233 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:58 smithi188 sudo[44989]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T02:53:59.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:58 smithi188 sudo[44989]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T02:53:59.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:58 smithi188 sudo[44989]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T02:53:59.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:58 smithi188 sudo[44989]: pam_unix(sudo:session): session closed for user root 2024-03-24T02:53:59.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:59 smithi188 sudo[44992]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T02:53:59.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:59 smithi188 sudo[44992]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T02:53:59.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:59 smithi188 sudo[44992]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T02:53:59.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:53:59 smithi188 sudo[44992]: pam_unix(sudo:session): session closed for user root 2024-03-24T02:54:00.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:59 smithi019 sudo[44264]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T02:54:00.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:59 smithi019 sudo[44264]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T02:54:00.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:59 smithi019 sudo[44264]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T02:54:00.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:53:59 smithi019 sudo[44264]: pam_unix(sudo:session): session closed for user root 2024-03-24T02:54:00.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:00 smithi017 ceph-mon[31926]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:00.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:00 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:00.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:00 smithi017 ceph-mon[31926]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T02:54:00.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:00 smithi017 ceph-mon[31926]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T02:54:00.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:00 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:54:00.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:00 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2912012041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:54:00.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:00 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:54:00.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:00 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:54:00.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:00 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:54:00.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:00 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:54:00.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:00 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:54:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:00 smithi188 ceph-mon[36206]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:00 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:00 smithi188 ceph-mon[36206]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T02:54:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:00 smithi188 ceph-mon[36206]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T02:54:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:00 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:54:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:00 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2912012041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:54:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:00 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:54:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:00 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:54:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:00 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:54:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:00 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:54:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:00 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:54:00.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 sudo[44267]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T02:54:00.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 sudo[44267]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T02:54:00.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 sudo[44267]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T02:54:00.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 sudo[44267]: pam_unix(sudo:session): session closed for user root 2024-03-24T02:54:00.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 ceph-mon[35944]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:00.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:00.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 ceph-mon[35944]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T02:54:00.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 ceph-mon[35944]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T02:54:00.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:54:00.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2912012041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:54:00.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:54:00.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:54:00.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:54:00.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:54:00.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:00 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:54:01.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:01 smithi017 ceph-mon[31926]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T02:54:01.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:01 smithi017 ceph-mon[31926]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T02:54:01.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:01 smithi017 ceph-mon[31926]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T02:54:01.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:54:01.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:54:01.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:54:01.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:01 smithi017 ceph-mon[31926]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:01.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:01 smithi017 ceph-mon[31926]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T02:54:01.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:01 smithi188 ceph-mon[36206]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T02:54:01.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:01 smithi188 ceph-mon[36206]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T02:54:01.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:01 smithi188 ceph-mon[36206]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T02:54:01.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:54:01.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:54:01.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:54:01.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:01 smithi188 ceph-mon[36206]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:01.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:01 smithi188 ceph-mon[36206]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T02:54:01.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:01 smithi019 ceph-mon[35944]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T02:54:01.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:01 smithi019 ceph-mon[35944]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T02:54:01.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:01 smithi019 ceph-mon[35944]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T02:54:01.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T02:54:01.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T02:54:01.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T02:54:01.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:01 smithi019 ceph-mon[35944]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:01.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:01 smithi019 ceph-mon[35944]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T02:54:02.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:02 smithi017 ceph-mon[31926]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:02.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:02 smithi188 ceph-mon[36206]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:02.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:02 smithi019 ceph-mon[35944]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:04.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:03 smithi017 ceph-mon[31926]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:04.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:03 smithi188 ceph-mon[36206]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:04.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:03 smithi019 ceph-mon[35944]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:06.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T02:54:06.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:06.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T02:54:06.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:06.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T02:54:06.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:06.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:06 smithi017 ceph-mon[31926]: Deploying daemon osd.3 on smithi017 2024-03-24T02:54:06.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:06 smithi017 ceph-mon[31926]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-03-24T02:54:07.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:06 smithi019 ceph-mon[35944]: Deploying daemon osd.3 on smithi017 2024-03-24T02:54:07.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:06 smithi019 ceph-mon[35944]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-03-24T02:54:07.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:06 smithi188 ceph-mon[36206]: Deploying daemon osd.3 on smithi017 2024-03-24T02:54:07.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:06 smithi188 ceph-mon[36206]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-03-24T02:54:08.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:07 smithi017 ceph-mon[31926]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-24T02:54:08.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:07 smithi188 ceph-mon[36206]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-24T02:54:08.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:07 smithi019 ceph-mon[35944]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-24T02:54:10.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:09 smithi017 ceph-mon[31926]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-24T02:54:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:09 smithi188 ceph-mon[36206]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-24T02:54:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:09 smithi019 ceph-mon[35944]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-24T02:54:12.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:12 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:12.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:12 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:12.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:12 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:54:12.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:12 smithi017 ceph-mon[31926]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-24T02:54:12.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:12 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:12.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:12 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:12.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:12 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:54:12.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:12 smithi188 ceph-mon[36206]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-24T02:54:12.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:12 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:12.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:12 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:12.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:12 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:54:12.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:12 smithi019 ceph-mon[35944]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-24T02:54:14.108 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:13 smithi019 ceph-mon[35944]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-24T02:54:14.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:13 smithi017 ceph-mon[31926]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-24T02:54:14.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:13 smithi188 ceph-mon[36206]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-24T02:54:15.234 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 3 on host 'smithi017' 2024-03-24T02:54:15.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.234+0000 7f8c7bfff700 1 -- 172.21.15.17:0/1007682930 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8c941a2d10 con 0x7f8c7c0a2880 2024-03-24T02:54:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.237+0000 7f8c9921c700 1 -- 172.21.15.17:0/1007682930 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8c7c0a2880 msgr2=0x7f8c7c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:54:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.237+0000 7f8c9921c700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8c7c0a2880 0x7f8c7c0a4d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8c880104e0 tx=0x7f8c8800b3c0 comp rx=0 tx=0).stop 2024-03-24T02:54:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.237+0000 7f8c9921c700 1 -- 172.21.15.17:0/1007682930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c941013b0 msgr2=0x7f8c94105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:54:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.237+0000 7f8c9921c700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c941013b0 0x7f8c94105100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400c010 tx=0x7f8c84009580 comp rx=0 tx=0).stop 2024-03-24T02:54:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.237+0000 7f8c9921c700 1 -- 172.21.15.17:0/1007682930 shutdown_connections 2024-03-24T02:54:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.237+0000 7f8c9921c700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8c7c0a2880 0x7f8c7c0a4d40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.237+0000 7f8c9921c700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c94102670 0x7f8c94103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.237+0000 7f8c9921c700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c941013b0 0x7f8c94105100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:15.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.237+0000 7f8c9921c700 1 --2- 172.21.15.17:0/1007682930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c94101d50 0x7f8c941037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:15.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.237+0000 7f8c9921c700 1 -- 172.21.15.17:0/1007682930 >> 172.21.15.17:0/1007682930 conn(0x7f8c940fbde0 msgr2=0x7f8c940fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:54:15.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.237+0000 7f8c9921c700 1 -- 172.21.15.17:0/1007682930 shutdown_connections 2024-03-24T02:54:15.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:54:15.237+0000 7f8c9921c700 1 -- 172.21.15.17:0/1007682930 wait complete. 2024-03-24T02:54:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:15 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:15 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:15 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:15 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:54:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:15 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:15 smithi017 ceph-mon[31926]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T02:54:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:15 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:15 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:15.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:54:15.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:15 smithi188 ceph-mon[36206]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T02:54:15.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:15 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:15 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:15 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:15.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:15 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:54:15.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:15 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:15 smithi019 ceph-mon[35944]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T02:54:15.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:15 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:15 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:15.960 DEBUG:teuthology.orchestra.run.smithi017:osd.3> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.3.service 2024-03-24T02:54:15.962 INFO:tasks.cephadm:Deploying osd.4 on smithi019 with /dev/vg_nvme/lv_3... 2024-03-24T02:54:15.962 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-24T02:54:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T02:54:16.454 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:16 smithi017 ceph-mon[31926]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T02:54:16.454 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:16 smithi017 ceph-mon[31926]: osdmap e23: 4 total, 3 up, 4 in 2024-03-24T02:54:16.454 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:16 smithi017 ceph-mon[31926]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-03-24T02:54:16.454 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:16 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:54:16.454 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:16 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:54:16.454 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:16 smithi017 ceph-mon[31926]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:16.506 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:16 smithi019 ceph-mon[35944]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T02:54:16.506 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:16 smithi019 ceph-mon[35944]: osdmap e23: 4 total, 3 up, 4 in 2024-03-24T02:54:16.506 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:16 smithi019 ceph-mon[35944]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-03-24T02:54:16.506 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:16 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:54:16.506 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:16 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:54:16.506 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:16 smithi019 ceph-mon[35944]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:16.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:16 smithi188 ceph-mon[36206]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T02:54:16.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:16 smithi188 ceph-mon[36206]: osdmap e23: 4 total, 3 up, 4 in 2024-03-24T02:54:16.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:16 smithi188 ceph-mon[36206]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-03-24T02:54:16.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:16 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:54:16.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:16 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:54:16.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:16 smithi188 ceph-mon[36206]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-03-24T02:54:17.206 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 02:54:16 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3[53083]: 2024-03-24T02:54:16.790+0000 7faffde3e700 -1 osd.3 0 waiting for initial osdmap 2024-03-24T02:54:17.206 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 02:54:16 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3[53083]: 2024-03-24T02:54:16.795+0000 7faff5428700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T02:54:17.502 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:17 smithi019 ceph-mon[35944]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-03-24T02:54:17.502 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:17 smithi019 ceph-mon[35944]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T02:54:17.502 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:54:17.502 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:54:17.502 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:17 smithi019 ceph-mon[35944]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' 2024-03-24T02:54:17.502 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:17 smithi019 ceph-mon[35944]: osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113] boot 2024-03-24T02:54:17.502 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:17 smithi019 ceph-mon[35944]: osdmap e25: 4 total, 4 up, 4 in 2024-03-24T02:54:17.503 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:54:17.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:17 smithi188 ceph-mon[36206]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-03-24T02:54:17.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:17 smithi188 ceph-mon[36206]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T02:54:17.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:54:17.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:54:17.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:17 smithi188 ceph-mon[36206]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' 2024-03-24T02:54:17.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:17 smithi188 ceph-mon[36206]: osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113] boot 2024-03-24T02:54:17.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:17 smithi188 ceph-mon[36206]: osdmap e25: 4 total, 4 up, 4 in 2024-03-24T02:54:17.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:54:17.560 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:17 smithi017 ceph-mon[31926]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-03-24T02:54:17.560 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:17 smithi017 ceph-mon[31926]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T02:54:17.560 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:54:17.560 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:54:17.560 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:17 smithi017 ceph-mon[31926]: from='osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113]' entity='osd.3' 2024-03-24T02:54:17.560 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:17 smithi017 ceph-mon[31926]: osd.3 [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113] boot 2024-03-24T02:54:17.561 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:17 smithi017 ceph-mon[31926]: osdmap e25: 4 total, 4 up, 4 in 2024-03-24T02:54:17.561 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T02:54:18.214 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-24T02:54:18.215 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-24T02:54:18.215 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-03-24T02:54:18.215 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-03-24T02:54:18.215 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0138004 s, 760 MB/s 2024-03-24T02:54:18.215 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 2024-03-24T02:54:18.215 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-03-24T02:54:18.532 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:18 smithi019 ceph-mon[35944]: purged_snaps scrub starts 2024-03-24T02:54:18.532 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:18 smithi019 ceph-mon[35944]: purged_snaps scrub ok 2024-03-24T02:54:18.533 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:18 smithi019 ceph-mon[35944]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:18.533 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:18 smithi019 ceph-mon[35944]: osdmap e26: 4 total, 4 up, 4 in 2024-03-24T02:54:18.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:18 smithi188 ceph-mon[36206]: purged_snaps scrub starts 2024-03-24T02:54:18.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:18 smithi188 ceph-mon[36206]: purged_snaps scrub ok 2024-03-24T02:54:18.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:18 smithi188 ceph-mon[36206]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:18.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:18 smithi188 ceph-mon[36206]: osdmap e26: 4 total, 4 up, 4 in 2024-03-24T02:54:18.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:18 smithi017 ceph-mon[31926]: purged_snaps scrub starts 2024-03-24T02:54:18.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:18 smithi017 ceph-mon[31926]: purged_snaps scrub ok 2024-03-24T02:54:18.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:18 smithi017 ceph-mon[31926]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:18.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:18 smithi017 ceph-mon[31926]: osdmap e26: 4 total, 4 up, 4 in 2024-03-24T02:54:18.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-03-24T02:54:19.102 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T02:54:20.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:20 smithi019 ceph-mon[35944]: Detected new or changed devices on smithi017 2024-03-24T02:54:20.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:20.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:20.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:54:20.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:54:20.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:20 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi017 to 8546M 2024-03-24T02:54:20.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:20.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:20.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:54:20.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:20.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:20 smithi019 ceph-mon[35944]: osdmap e27: 4 total, 4 up, 4 in 2024-03-24T02:54:20.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:20 smithi019 ceph-mon[35944]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:20.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:20 smithi017 ceph-mon[31926]: Detected new or changed devices on smithi017 2024-03-24T02:54:20.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:20.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:20.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:54:20.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:54:20.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:20 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi017 to 8546M 2024-03-24T02:54:20.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:20.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:20.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:54:20.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:20.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:20 smithi017 ceph-mon[31926]: osdmap e27: 4 total, 4 up, 4 in 2024-03-24T02:54:20.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:20 smithi017 ceph-mon[31926]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:20.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:20 smithi188 ceph-mon[36206]: Detected new or changed devices on smithi017 2024-03-24T02:54:20.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:20.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:20.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:54:20.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:54:20.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:20 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi017 to 8546M 2024-03-24T02:54:20.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:20.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:20.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:54:20.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:20.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:20 smithi188 ceph-mon[36206]: osdmap e27: 4 total, 4 up, 4 in 2024-03-24T02:54:20.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:20 smithi188 ceph-mon[36206]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:20.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.880+0000 7f687b693700 1 -- 172.21.15.19:0/2082328791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874105df0 msgr2=0x7f6874106270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:54:20.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.880+0000 7f687b693700 1 --2- 172.21.15.19:0/2082328791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874105df0 0x7f6874106270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f686c009990 tx=0x7f686c009ca0 comp rx=0 tx=0).stop 2024-03-24T02:54:20.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.880+0000 7f687b693700 1 -- 172.21.15.19:0/2082328791 shutdown_connections 2024-03-24T02:54:20.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.880+0000 7f687b693700 1 --2- 172.21.15.19:0/2082328791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874105df0 0x7f6874106270 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:20.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.880+0000 7f687b693700 1 --2- 172.21.15.19:0/2082328791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6874100ab0 0x7f6874102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:20.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.880+0000 7f687b693700 1 --2- 172.21.15.19:0/2082328791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6874103420 0x7f68741058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:20.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.880+0000 7f687b693700 1 -- 172.21.15.19:0/2082328791 >> 172.21.15.19:0/2082328791 conn(0x7f68740fc6f0 msgr2=0x7f68740feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:54:20.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.880+0000 7f687b693700 1 -- 172.21.15.19:0/2082328791 shutdown_connections 2024-03-24T02:54:20.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.880+0000 7f687b693700 1 -- 172.21.15.19:0/2082328791 wait complete. 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f687b693700 1 Processor -- start 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f687b693700 1 -- start start 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f687b693700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6874100ab0 0x7f687419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f687b693700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6874103420 0x7f687419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f687b693700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874105df0 0x7f68741a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f687b693700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6874114320 con 0x7f6874103420 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f687b693700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68741141a0 con 0x7f6874105df0 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f687b693700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68741144a0 con 0x7f6874100ab0 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f6879c30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874105df0 0x7f68741a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f6879c30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874105df0 0x7f68741a5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59246/0 (socket says 172.21.15.19:59246) 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f6879c30700 1 -- 172.21.15.19:0/1134597998 learned_addr learned my addr 172.21.15.19:0/1134597998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f687942f700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6874100ab0 0x7f687419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f6878c2e700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6874103420 0x7f687419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:54:20.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f6879c30700 1 -- 172.21.15.19:0/1134597998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6874100ab0 msgr2=0x7f687419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:54:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f6879c30700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6874100ab0 0x7f687419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f6879c30700 1 -- 172.21.15.19:0/1134597998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6874103420 msgr2=0x7f687419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:54:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f6879c30700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6874103420 0x7f687419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f6879c30700 1 -- 172.21.15.19:0/1134597998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68741a6540 con 0x7f6874105df0 2024-03-24T02:54:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f6878c2e700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6874103420 0x7f687419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:54:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.881+0000 7f6879c30700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874105df0 0x7f68741a5ea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f686c0039c0 tx=0x7f686c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:54:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.882+0000 7f686a7fc700 1 -- 172.21.15.19:0/1134597998 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c027070 con 0x7f6874105df0 2024-03-24T02:54:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.882+0000 7f687b693700 1 -- 172.21.15.19:0/1134597998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68741a67d0 con 0x7f6874105df0 2024-03-24T02:54:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.882+0000 7f686a7fc700 1 -- 172.21.15.19:0/1134597998 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f686c004150 con 0x7f6874105df0 2024-03-24T02:54:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.882+0000 7f686a7fc700 1 -- 172.21.15.19:0/1134597998 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c00f460 con 0x7f6874105df0 2024-03-24T02:54:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.882+0000 7f687b693700 1 -- 172.21.15.19:0/1134597998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68741a6ce0 con 0x7f6874105df0 2024-03-24T02:54:20.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.883+0000 7f687b693700 1 -- 172.21.15.19:0/1134597998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6874068e50 con 0x7f6874105df0 2024-03-24T02:54:20.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.884+0000 7f686a7fc700 1 -- 172.21.15.19:0/1134597998 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f686c02a070 con 0x7f6874105df0 2024-03-24T02:54:20.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.885+0000 7f686a7fc700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f68600a2890 0x7f68600a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:54:20.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.885+0000 7f686a7fc700 1 -- 172.21.15.19:0/1134597998 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3669+0+0 (secure 0 0 0) 0x7f686c0cf0a0 con 0x7f6874105df0 2024-03-24T02:54:20.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.885+0000 7f687942f700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f68600a2890 0x7f68600a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:54:20.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.886+0000 7f687942f700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f68600a2890 0x7f68600a4d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6870009d10 tx=0x7f6870009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:54:20.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:20.889+0000 7f686a7fc700 1 -- 172.21.15.19:0/1134597998 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f686c013070 con 0x7f6874105df0 2024-03-24T02:54:21.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:21.105+0000 7f687b693700 1 -- 172.21.15.19:0/1134597998 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6874062690 con 0x7f68600a2890 2024-03-24T02:54:21.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:21 smithi188 ceph-mon[36206]: osdmap e28: 4 total, 4 up, 4 in 2024-03-24T02:54:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:21 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:54:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:21 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:54:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:21 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:21 smithi019 ceph-mon[35944]: osdmap e28: 4 total, 4 up, 4 in 2024-03-24T02:54:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:21 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:54:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:21 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:54:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:21 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:21.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:21 smithi017 ceph-mon[31926]: osdmap e28: 4 total, 4 up, 4 in 2024-03-24T02:54:21.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:21 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:54:21.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:21 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:54:21.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:21 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:22.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:22 smithi188 ceph-mon[36206]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:54:22.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:22 smithi188 ceph-mon[36206]: osdmap e29: 4 total, 4 up, 4 in 2024-03-24T02:54:22.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:22 smithi188 ceph-mon[36206]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-24T02:54:22.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:22 smithi019 ceph-mon[35944]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:54:22.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:22 smithi019 ceph-mon[35944]: osdmap e29: 4 total, 4 up, 4 in 2024-03-24T02:54:22.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:22 smithi019 ceph-mon[35944]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-24T02:54:22.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:22 smithi017 ceph-mon[31926]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:54:22.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:22 smithi017 ceph-mon[31926]: osdmap e29: 4 total, 4 up, 4 in 2024-03-24T02:54:22.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:22 smithi017 ceph-mon[31926]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-24T02:54:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:23 smithi017 ceph-mon[31926]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-24T02:54:24.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:23 smithi188 ceph-mon[36206]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-24T02:54:24.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:23 smithi019 ceph-mon[35944]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-24T02:54:25.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:24 smithi017 ceph-mon[31926]: from='client.? 172.21.15.19:0/1068764636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "659ccfee-f6c0-4d35-b557-f7978d553346"}]: dispatch 2024-03-24T02:54:25.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:24 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "659ccfee-f6c0-4d35-b557-f7978d553346"}]: dispatch 2024-03-24T02:54:25.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:24 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "659ccfee-f6c0-4d35-b557-f7978d553346"}]': finished 2024-03-24T02:54:25.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:24 smithi017 ceph-mon[31926]: osdmap e30: 5 total, 4 up, 5 in 2024-03-24T02:54:25.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:24 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:25.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:24 smithi017 ceph-mon[31926]: from='client.? 172.21.15.19:0/2641041231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:54:25.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:24 smithi188 ceph-mon[36206]: from='client.? 172.21.15.19:0/1068764636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "659ccfee-f6c0-4d35-b557-f7978d553346"}]: dispatch 2024-03-24T02:54:25.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:24 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "659ccfee-f6c0-4d35-b557-f7978d553346"}]: dispatch 2024-03-24T02:54:25.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:24 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "659ccfee-f6c0-4d35-b557-f7978d553346"}]': finished 2024-03-24T02:54:25.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:24 smithi188 ceph-mon[36206]: osdmap e30: 5 total, 4 up, 5 in 2024-03-24T02:54:25.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:24 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:25.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:24 smithi188 ceph-mon[36206]: from='client.? 172.21.15.19:0/2641041231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:54:25.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:24 smithi019 ceph-mon[35944]: from='client.? 172.21.15.19:0/1068764636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "659ccfee-f6c0-4d35-b557-f7978d553346"}]: dispatch 2024-03-24T02:54:25.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:24 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "659ccfee-f6c0-4d35-b557-f7978d553346"}]: dispatch 2024-03-24T02:54:25.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:24 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "659ccfee-f6c0-4d35-b557-f7978d553346"}]': finished 2024-03-24T02:54:25.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:24 smithi019 ceph-mon[35944]: osdmap e30: 5 total, 4 up, 5 in 2024-03-24T02:54:25.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:24 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:25.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:24 smithi019 ceph-mon[35944]: from='client.? 172.21.15.19:0/2641041231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:54:26.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:25 smithi188 ceph-mon[36206]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T02:54:26.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:25 smithi019 ceph-mon[35944]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T02:54:26.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:25 smithi017 ceph-mon[31926]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T02:54:28.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:27 smithi017 ceph-mon[31926]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:28.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:27 smithi188 ceph-mon[36206]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:28.298 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:27 smithi019 ceph-mon[35944]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:30.188 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:29 smithi019 ceph-mon[35944]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:30.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:29 smithi017 ceph-mon[31926]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:30.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:29 smithi188 ceph-mon[36206]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:31.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T02:54:31.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:31.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T02:54:31.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:31.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T02:54:31.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:32.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:32 smithi019 ceph-mon[35944]: Deploying daemon osd.4 on smithi019 2024-03-24T02:54:32.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:32 smithi019 ceph-mon[35944]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:32.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:32 smithi017 ceph-mon[31926]: Deploying daemon osd.4 on smithi019 2024-03-24T02:54:32.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:32 smithi017 ceph-mon[31926]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:32.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:32 smithi188 ceph-mon[36206]: Deploying daemon osd.4 on smithi019 2024-03-24T02:54:32.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:32 smithi188 ceph-mon[36206]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:33 smithi017 ceph-mon[31926]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:34.245 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:33 smithi019 ceph-mon[35944]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:34.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:33 smithi188 ceph-mon[36206]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:36.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:35 smithi019 ceph-mon[35944]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:35 smithi017 ceph-mon[31926]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:35 smithi188 ceph-mon[36206]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:38.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:37 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:38.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:37 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:38.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:37 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:54:38.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:37 smithi188 ceph-mon[36206]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:37 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:38.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:37 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:38.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:37 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:54:38.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:37 smithi019 ceph-mon[35944]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:37 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:37 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:37 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:54:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:37 smithi017 ceph-mon[31926]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:40.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:39 smithi017 ceph-mon[31926]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:40.216 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:39 smithi019 ceph-mon[35944]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:40.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.272+0000 7f686a7fc700 1 -- 172.21.15.19:0/1134597998 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6874062690 con 0x7f68600a2890 2024-03-24T02:54:40.273 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 4 on host 'smithi019' 2024-03-24T02:54:40.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.274+0000 7f687b693700 1 -- 172.21.15.19:0/1134597998 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f68600a2890 msgr2=0x7f68600a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:54:40.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.274+0000 7f687b693700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f68600a2890 0x7f68600a4d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6870009d10 tx=0x7f6870009040 comp rx=0 tx=0).stop 2024-03-24T02:54:40.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.274+0000 7f687b693700 1 -- 172.21.15.19:0/1134597998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874105df0 msgr2=0x7f68741a5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:54:40.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.274+0000 7f687b693700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874105df0 0x7f68741a5ea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f686c0039c0 tx=0x7f686c005e80 comp rx=0 tx=0).stop 2024-03-24T02:54:40.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.274+0000 7f687b693700 1 -- 172.21.15.19:0/1134597998 shutdown_connections 2024-03-24T02:54:40.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.274+0000 7f687b693700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f68600a2890 0x7f68600a4d50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:40.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.274+0000 7f687b693700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6874105df0 0x7f68741a5ea0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:40.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.274+0000 7f687b693700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6874100ab0 0x7f687419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:40.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.274+0000 7f687b693700 1 --2- 172.21.15.19:0/1134597998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6874103420 0x7f687419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:40.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.274+0000 7f687b693700 1 -- 172.21.15.19:0/1134597998 >> 172.21.15.19:0/1134597998 conn(0x7f68740fc6f0 msgr2=0x7f6874101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:54:40.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.274+0000 7f687b693700 1 -- 172.21.15.19:0/1134597998 shutdown_connections 2024-03-24T02:54:40.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:54:40.274+0000 7f687b693700 1 -- 172.21.15.19:0/1134597998 wait complete. 2024-03-24T02:54:40.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:39 smithi188 ceph-mon[36206]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:40.908 DEBUG:teuthology.orchestra.run.smithi019:osd.4> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.4.service 2024-03-24T02:54:40.911 INFO:tasks.cephadm:Deploying osd.5 on smithi188 with /dev/vg_nvme/lv_3... 2024-03-24T02:54:40.911 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-24T02:54:41.157 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T02:54:41.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:41.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:41.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:41 smithi188 ceph-mon[36206]: from='osd.4 [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T02:54:41.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:41 smithi188 ceph-mon[36206]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T02:54:41.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:41.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:41.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:41.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:41.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:41 smithi019 ceph-mon[35944]: from='osd.4 [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T02:54:41.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:41 smithi019 ceph-mon[35944]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T02:54:41.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:41.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:41.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:41.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:41.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:41 smithi017 ceph-mon[31926]: from='osd.4 [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T02:54:41.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:41 smithi017 ceph-mon[31926]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T02:54:41.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:41.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:42 smithi188 ceph-mon[36206]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T02:54:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:42 smithi188 ceph-mon[36206]: osdmap e31: 5 total, 4 up, 5 in 2024-03-24T02:54:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:42 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:42 smithi188 ceph-mon[36206]: from='osd.4 [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:54:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:42 smithi188 ceph-mon[36206]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:54:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:42 smithi188 ceph-mon[36206]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:42.550 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:42 smithi019 ceph-mon[35944]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T02:54:42.550 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:42 smithi019 ceph-mon[35944]: osdmap e31: 5 total, 4 up, 5 in 2024-03-24T02:54:42.550 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:42 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:42.550 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:42 smithi019 ceph-mon[35944]: from='osd.4 [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:54:42.550 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:42 smithi019 ceph-mon[35944]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:54:42.551 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:42 smithi019 ceph-mon[35944]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:42.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:42 smithi017 ceph-mon[31926]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T02:54:42.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:42 smithi017 ceph-mon[31926]: osdmap e31: 5 total, 4 up, 5 in 2024-03-24T02:54:42.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:42 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:42.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:42 smithi017 ceph-mon[31926]: from='osd.4 [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:54:42.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:42 smithi017 ceph-mon[31926]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:54:42.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:42 smithi017 ceph-mon[31926]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:43.138 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 02:54:42 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4[47074]: 2024-03-24T02:54:42.976+0000 7fb780d4f700 -1 osd.4 0 waiting for initial osdmap 2024-03-24T02:54:43.138 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 02:54:42 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4[47074]: 2024-03-24T02:54:42.985+0000 7fb778b3a700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T02:54:43.197 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-24T02:54:43.197 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-24T02:54:43.197 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-03-24T02:54:43.197 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-03-24T02:54:43.197 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111268 s, 942 MB/s 2024-03-24T02:54:43.197 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-03-24T02:54:43.500 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:43 smithi188 ceph-mon[36206]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-24T02:54:43.500 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:43 smithi188 ceph-mon[36206]: osdmap e32: 5 total, 4 up, 5 in 2024-03-24T02:54:43.501 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:43 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:43.501 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:43 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:43.501 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:43 smithi188 ceph-mon[36206]: from='osd.4 ' entity='osd.4' 2024-03-24T02:54:43.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:43 smithi019 ceph-mon[35944]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-24T02:54:43.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:43 smithi019 ceph-mon[35944]: osdmap e32: 5 total, 4 up, 5 in 2024-03-24T02:54:43.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:43 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:43.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:43 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:43.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:43 smithi019 ceph-mon[35944]: from='osd.4 ' entity='osd.4' 2024-03-24T02:54:43.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:43 smithi017 ceph-mon[31926]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-24T02:54:43.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:43 smithi017 ceph-mon[31926]: osdmap e32: 5 total, 4 up, 5 in 2024-03-24T02:54:43.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:43 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:43.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:43 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:43.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:43 smithi017 ceph-mon[31926]: from='osd.4 ' entity='osd.4' 2024-03-24T02:54:43.828 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi188:vg_nvme/lv_3 2024-03-24T02:54:44.068 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T02:54:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:44 smithi019 ceph-mon[35944]: purged_snaps scrub starts 2024-03-24T02:54:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:44 smithi019 ceph-mon[35944]: purged_snaps scrub ok 2024-03-24T02:54:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:44.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:44 smithi019 ceph-mon[35944]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:44.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:44 smithi019 ceph-mon[35944]: osd.4 [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962] boot 2024-03-24T02:54:44.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:44 smithi019 ceph-mon[35944]: osdmap e33: 5 total, 5 up, 5 in 2024-03-24T02:54:44.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:44.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:44 smithi188 ceph-mon[36206]: purged_snaps scrub starts 2024-03-24T02:54:44.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:44 smithi188 ceph-mon[36206]: purged_snaps scrub ok 2024-03-24T02:54:44.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:44.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:44 smithi188 ceph-mon[36206]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:44.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:44 smithi188 ceph-mon[36206]: osd.4 [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962] boot 2024-03-24T02:54:44.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:44 smithi188 ceph-mon[36206]: osdmap e33: 5 total, 5 up, 5 in 2024-03-24T02:54:44.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:44.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:44 smithi017 ceph-mon[31926]: purged_snaps scrub starts 2024-03-24T02:54:44.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:44 smithi017 ceph-mon[31926]: purged_snaps scrub ok 2024-03-24T02:54:44.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:44.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:44 smithi017 ceph-mon[31926]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-24T02:54:44.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:44 smithi017 ceph-mon[31926]: osd.4 [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962] boot 2024-03-24T02:54:44.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:44 smithi017 ceph-mon[31926]: osdmap e33: 5 total, 5 up, 5 in 2024-03-24T02:54:44.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T02:54:45.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.564+0000 7f7a99f9a700 1 -- 172.21.15.188:0/909694343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a94102690 msgr2=0x7f7a94102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:54:45.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.564+0000 7f7a99f9a700 1 --2- 172.21.15.188:0/909694343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a94102690 0x7f7a94102ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7a840099c0 tx=0x7f7a84009cd0 comp rx=0 tx=0).stop 2024-03-24T02:54:45.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.564+0000 7f7a99f9a700 1 -- 172.21.15.188:0/909694343 shutdown_connections 2024-03-24T02:54:45.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.565+0000 7f7a99f9a700 1 --2- 172.21.15.188:0/909694343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a94101d70 0x7f7a94109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:45.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.565+0000 7f7a99f9a700 1 --2- 172.21.15.188:0/909694343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a94102690 0x7f7a94102ab0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.565+0000 7f7a99f9a700 1 --2- 172.21.15.188:0/909694343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a941013d0 0x7f7a94101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.565+0000 7f7a99f9a700 1 -- 172.21.15.188:0/909694343 >> 172.21.15.188:0/909694343 conn(0x7f7a940fbe20 msgr2=0x7f7a940fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.565+0000 7f7a99f9a700 1 -- 172.21.15.188:0/909694343 shutdown_connections 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.565+0000 7f7a99f9a700 1 -- 172.21.15.188:0/909694343 wait complete. 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.565+0000 7f7a99f9a700 1 Processor -- start 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.565+0000 7f7a99f9a700 1 -- start start 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.565+0000 7f7a99f9a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a941013d0 0x7f7a94100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.565+0000 7f7a99f9a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a94101d70 0x7f7a9419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.565+0000 7f7a99f9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a94102690 0x7f7a9419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a99f9a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9410cdc0 con 0x7f7a94102690 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a99f9a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9410cc40 con 0x7f7a94101d70 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a99f9a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9410cf40 con 0x7f7a941013d0 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a937fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a941013d0 0x7f7a94100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:54:45.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a937fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a941013d0 0x7f7a94100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:38702/0 (socket says 172.21.15.188:38702) 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a937fe700 1 -- 172.21.15.188:0/1257267602 learned_addr learned my addr 172.21.15.188:0/1257267602 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a92ffd700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a94101d70 0x7f7a9419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a937fe700 1 -- 172.21.15.188:0/1257267602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a94101d70 msgr2=0x7f7a9419d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a937fe700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a94101d70 0x7f7a9419d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a937fe700 1 -- 172.21.15.188:0/1257267602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a94102690 msgr2=0x7f7a9419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a937fe700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a94102690 0x7f7a9419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a937fe700 1 -- 172.21.15.188:0/1257267602 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a941a0180 con 0x7f7a941013d0 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a937fe700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a941013d0 0x7f7a94100e20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400c010 tx=0x7f7a84005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a90ff9700 1 -- 172.21.15.188:0/1257267602 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84027070 con 0x7f7a941013d0 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a99f9a700 1 -- 172.21.15.188:0/1257267602 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a941a0410 con 0x7f7a941013d0 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a90ff9700 1 -- 172.21.15.188:0/1257267602 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a840041d0 con 0x7f7a941013d0 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a90ff9700 1 -- 172.21.15.188:0/1257267602 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8400f460 con 0x7f7a941013d0 2024-03-24T02:54:45.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.566+0000 7f7a99f9a700 1 -- 172.21.15.188:0/1257267602 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a941a0970 con 0x7f7a941013d0 2024-03-24T02:54:45.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.567+0000 7f7a99f9a700 1 -- 172.21.15.188:0/1257267602 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a94068e50 con 0x7f7a941013d0 2024-03-24T02:54:45.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.569+0000 7f7a90ff9700 1 -- 172.21.15.188:0/1257267602 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7a8402a070 con 0x7f7a941013d0 2024-03-24T02:54:45.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.569+0000 7f7a90ff9700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7a7c0a2890 0x7f7a7c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:54:45.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.570+0000 7f7a90ff9700 1 -- 172.21.15.188:0/1257267602 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f7a840cf130 con 0x7f7a941013d0 2024-03-24T02:54:45.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.570+0000 7f7a92ffd700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7a7c0a2890 0x7f7a7c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:54:45.570 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.570+0000 7f7a92ffd700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7a7c0a2890 0x7f7a7c0a4d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a941013a0 tx=0x7f7a88009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:54:45.573 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.573+0000 7f7a90ff9700 1 -- 172.21.15.188:0/1257267602 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7a84031070 con 0x7f7a941013d0 2024-03-24T02:54:45.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: Detected new or changed devices on smithi019 2024-03-24T02:54:45.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:54:45.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:54:45.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi019 to 8542M 2024-03-24T02:54:45.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:45.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:54:45.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:54:45.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:45.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:54:45.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:45 smithi017 ceph-mon[31926]: osdmap e34: 5 total, 5 up, 5 in 2024-03-24T02:54:45.775 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:54:45.776+0000 7f7a99f9a700 1 -- 172.21.15.188:0/1257267602 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a940625d0 con 0x7f7a7c0a2890 2024-03-24T02:54:45.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: Detected new or changed devices on smithi019 2024-03-24T02:54:45.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:54:45.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:54:45.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi019 to 8542M 2024-03-24T02:54:45.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:45.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:54:45.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:54:45.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:45.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:54:45.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:45 smithi188 ceph-mon[36206]: osdmap e34: 5 total, 5 up, 5 in 2024-03-24T02:54:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: Detected new or changed devices on smithi019 2024-03-24T02:54:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:54:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:54:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi019 to 8542M 2024-03-24T02:54:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:54:45.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:54:45.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:45.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:54:45.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:54:45.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:45 smithi019 ceph-mon[35944]: osdmap e34: 5 total, 5 up, 5 in 2024-03-24T02:54:46.672 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:46 smithi188 ceph-mon[36206]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:54:46.673 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:46 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:54:46.673 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:46 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:54:46.673 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:46 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:46.673 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:46 smithi188 ceph-mon[36206]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:46.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:46 smithi017 ceph-mon[31926]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:54:46.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:46 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:54:46.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:46 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:54:46.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:46 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:46.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:46 smithi017 ceph-mon[31926]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:46.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:46 smithi019 ceph-mon[35944]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:54:46.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:46 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:54:46.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:46 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:54:46.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:46 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:46.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:46 smithi019 ceph-mon[35944]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:48.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:47 smithi017 ceph-mon[31926]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:48.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:47 smithi188 ceph-mon[36206]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:48.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:47 smithi019 ceph-mon[35944]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:49.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:48 smithi188 ceph-mon[36206]: from='client.? 172.21.15.188:0/328058108' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3ec6587-2c61-4188-b4af-9f36ad6c4abc"}]: dispatch 2024-03-24T02:54:49.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:48 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3ec6587-2c61-4188-b4af-9f36ad6c4abc"}]: dispatch 2024-03-24T02:54:49.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:48 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3ec6587-2c61-4188-b4af-9f36ad6c4abc"}]': finished 2024-03-24T02:54:49.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:48 smithi188 ceph-mon[36206]: osdmap e35: 6 total, 5 up, 6 in 2024-03-24T02:54:49.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:48 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:54:49.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:48 smithi017 ceph-mon[31926]: from='client.? 172.21.15.188:0/328058108' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3ec6587-2c61-4188-b4af-9f36ad6c4abc"}]: dispatch 2024-03-24T02:54:49.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:48 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3ec6587-2c61-4188-b4af-9f36ad6c4abc"}]: dispatch 2024-03-24T02:54:49.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:48 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3ec6587-2c61-4188-b4af-9f36ad6c4abc"}]': finished 2024-03-24T02:54:49.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:48 smithi017 ceph-mon[31926]: osdmap e35: 6 total, 5 up, 6 in 2024-03-24T02:54:49.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:48 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:54:49.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:48 smithi019 ceph-mon[35944]: from='client.? 172.21.15.188:0/328058108' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3ec6587-2c61-4188-b4af-9f36ad6c4abc"}]: dispatch 2024-03-24T02:54:49.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:48 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3ec6587-2c61-4188-b4af-9f36ad6c4abc"}]: dispatch 2024-03-24T02:54:49.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:48 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3ec6587-2c61-4188-b4af-9f36ad6c4abc"}]': finished 2024-03-24T02:54:49.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:48 smithi019 ceph-mon[35944]: osdmap e35: 6 total, 5 up, 6 in 2024-03-24T02:54:49.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:48 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:54:50.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:49 smithi188 ceph-mon[36206]: from='client.? 172.21.15.188:0/1621232470' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:54:50.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:49 smithi188 ceph-mon[36206]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:49 smithi019 ceph-mon[35944]: from='client.? 172.21.15.188:0/1621232470' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:54:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:49 smithi019 ceph-mon[35944]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:50.455 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:49 smithi017 ceph-mon[31926]: from='client.? 172.21.15.188:0/1621232470' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:54:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:49 smithi017 ceph-mon[31926]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:52.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:51 smithi017 ceph-mon[31926]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:52.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:51 smithi188 ceph-mon[36206]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:51 smithi019 ceph-mon[35944]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:54.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:53 smithi017 ceph-mon[31926]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:54.292 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:53 smithi188 ceph-mon[36206]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:54.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:53 smithi019 ceph-mon[35944]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:56.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:55 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T02:54:56.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:55 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:56.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:55 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T02:54:56.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:55 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:56.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:55 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T02:54:56.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:55 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:54:57.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:56 smithi188 ceph-mon[36206]: Deploying daemon osd.5 on smithi188 2024-03-24T02:54:57.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:56 smithi188 ceph-mon[36206]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:57.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:56 smithi017 ceph-mon[31926]: Deploying daemon osd.5 on smithi188 2024-03-24T02:54:57.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:56 smithi017 ceph-mon[31926]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:56 smithi019 ceph-mon[35944]: Deploying daemon osd.5 on smithi188 2024-03-24T02:54:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:56 smithi019 ceph-mon[35944]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:58.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:57 smithi017 ceph-mon[31926]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:58.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:57 smithi188 ceph-mon[36206]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:54:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:57 smithi019 ceph-mon[35944]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:54:59 smithi017 ceph-mon[31926]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:00.228 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:54:59 smithi188 ceph-mon[36206]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:00.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:54:59 smithi019 ceph-mon[35944]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:02.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:02 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:02.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:02 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:02.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:02 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:02.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:02 smithi017 ceph-mon[31926]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:02.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:02 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:02.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:02 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:02.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:02 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:02.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:02 smithi188 ceph-mon[36206]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:02.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:02 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:02.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:02 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:02.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:02 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:02.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:02 smithi019 ceph-mon[35944]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:04.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:03 smithi188 ceph-mon[36206]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:04.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:03 smithi019 ceph-mon[35944]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:04.443 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.442+0000 7f7a90ff9700 1 -- 172.21.15.188:0/1257267602 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7a940625d0 con 0x7f7a7c0a2890 2024-03-24T02:55:04.443 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 5 on host 'smithi188' 2024-03-24T02:55:04.445 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.445+0000 7f7a99f9a700 1 -- 172.21.15.188:0/1257267602 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7a7c0a2890 msgr2=0x7f7a7c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:55:04.445 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.445+0000 7f7a99f9a700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7a7c0a2890 0x7f7a7c0a4d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a941013a0 tx=0x7f7a88009190 comp rx=0 tx=0).stop 2024-03-24T02:55:04.445 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.445+0000 7f7a99f9a700 1 -- 172.21.15.188:0/1257267602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a941013d0 msgr2=0x7f7a94100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:55:04.445 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.445+0000 7f7a99f9a700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a941013d0 0x7f7a94100e20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400c010 tx=0x7f7a84005e80 comp rx=0 tx=0).stop 2024-03-24T02:55:04.445 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.445+0000 7f7a99f9a700 1 -- 172.21.15.188:0/1257267602 shutdown_connections 2024-03-24T02:55:04.445 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.445+0000 7f7a99f9a700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7a7c0a2890 0x7f7a7c0a4d50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:04.445 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.445+0000 7f7a99f9a700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a94102690 0x7f7a9419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:04.445 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.445+0000 7f7a99f9a700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a941013d0 0x7f7a94100e20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:04.445 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.445+0000 7f7a99f9a700 1 --2- 172.21.15.188:0/1257267602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a94101d70 0x7f7a9419d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:04.446 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.445+0000 7f7a99f9a700 1 -- 172.21.15.188:0/1257267602 >> 172.21.15.188:0/1257267602 conn(0x7f7a940fbe20 msgr2=0x7f7a94105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:55:04.446 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.445+0000 7f7a99f9a700 1 -- 172.21.15.188:0/1257267602 shutdown_connections 2024-03-24T02:55:04.446 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:55:04.445+0000 7f7a99f9a700 1 -- 172.21.15.188:0/1257267602 wait complete. 2024-03-24T02:55:04.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:03 smithi017 ceph-mon[31926]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:05.071 DEBUG:teuthology.orchestra.run.smithi188:osd.5> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.5.service 2024-03-24T02:55:05.074 INFO:tasks.cephadm:Deploying osd.6 on smithi017 with /dev/vg_nvme/lv_2... 2024-03-24T02:55:05.074 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-24T02:55:05.326 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:55:05.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:05.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:05.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:05.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:05.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:05 smithi188 ceph-mon[36206]: from='osd.5 [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T02:55:05.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:05 smithi188 ceph-mon[36206]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T02:55:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:05 smithi017 ceph-mon[31926]: from='osd.5 [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T02:55:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:05 smithi017 ceph-mon[31926]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T02:55:05.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:05.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:05.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:05.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:05.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:05 smithi019 ceph-mon[35944]: from='osd.5 [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T02:55:05.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:05 smithi019 ceph-mon[35944]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T02:55:06.855 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:06 smithi017 ceph-mon[31926]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T02:55:06.855 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:06 smithi017 ceph-mon[31926]: osdmap e36: 6 total, 5 up, 6 in 2024-03-24T02:55:06.855 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:06.855 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:06 smithi017 ceph-mon[31926]: from='osd.5 [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:55:06.855 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:06 smithi017 ceph-mon[31926]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:55:06.856 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:06 smithi017 ceph-mon[31926]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:07.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:06 smithi188 ceph-mon[36206]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T02:55:07.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:06 smithi188 ceph-mon[36206]: osdmap e36: 6 total, 5 up, 6 in 2024-03-24T02:55:07.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:06 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:07.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:06 smithi188 ceph-mon[36206]: from='osd.5 [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:55:07.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:06 smithi188 ceph-mon[36206]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:55:07.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:06 smithi188 ceph-mon[36206]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:07.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:06 smithi019 ceph-mon[35944]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T02:55:07.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:06 smithi019 ceph-mon[35944]: osdmap e36: 6 total, 5 up, 6 in 2024-03-24T02:55:07.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:06 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:07.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:06 smithi019 ceph-mon[35944]: from='osd.5 [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:55:07.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:06 smithi019 ceph-mon[35944]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:55:07.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:06 smithi019 ceph-mon[35944]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:07.554 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-24T02:55:07.554 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-24T02:55:07.554 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2024-03-24T02:55:07.554 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2024-03-24T02:55:07.554 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00834615 s, 1.3 GB/s 2024-03-24T02:55:07.554 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2024-03-24T02:55:07.745 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 02:55:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T02:55:07.454+0000 7fc43570f700 -1 osd.5 0 waiting for initial osdmap 2024-03-24T02:55:07.745 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 02:55:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T02:55:07.462+0000 7fc42d4fa700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T02:55:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:07 smithi017 ceph-mon[31926]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-03-24T02:55:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:07 smithi017 ceph-mon[31926]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T02:55:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:07 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:07 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:07 smithi017 ceph-mon[31926]: from='osd.5 ' entity='osd.5' 2024-03-24T02:55:08.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:07 smithi188 ceph-mon[36206]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-03-24T02:55:08.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:07 smithi188 ceph-mon[36206]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T02:55:08.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:07 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:08.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:07 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:08.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:07 smithi188 ceph-mon[36206]: from='osd.5 ' entity='osd.5' 2024-03-24T02:55:08.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:07 smithi019 ceph-mon[35944]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-03-24T02:55:08.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:07 smithi019 ceph-mon[35944]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T02:55:08.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:07 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:08.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:07 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:08.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:07 smithi019 ceph-mon[35944]: from='osd.5 ' entity='osd.5' 2024-03-24T02:55:08.400 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi017:vg_nvme/lv_2 2024-03-24T02:55:08.640 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:55:09.015 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: purged_snaps scrub starts 2024-03-24T02:55:09.015 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: purged_snaps scrub ok 2024-03-24T02:55:09.015 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: osd.5 [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360] boot 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: osdmap e38: 6 total, 6 up, 6 in 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:09.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:55:09.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: purged_snaps scrub starts 2024-03-24T02:55:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: purged_snaps scrub ok 2024-03-24T02:55:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: osd.5 [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360] boot 2024-03-24T02:55:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: osdmap e38: 6 total, 6 up, 6 in 2024-03-24T02:55:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:09.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:55:09.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:09.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:09.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:55:09.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: purged_snaps scrub starts 2024-03-24T02:55:09.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: purged_snaps scrub ok 2024-03-24T02:55:09.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:09.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: osd.5 [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360] boot 2024-03-24T02:55:09.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: osdmap e38: 6 total, 6 up, 6 in 2024-03-24T02:55:09.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:55:09.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:09.945 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:09 smithi017 ceph-mon[31926]: Detected new or changed devices on smithi188 2024-03-24T02:55:09.946 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:09 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi188 to 8540M 2024-03-24T02:55:09.946 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:09 smithi017 ceph-mon[31926]: osdmap e39: 6 total, 6 up, 6 in 2024-03-24T02:55:10.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:09 smithi188 ceph-mon[36206]: Detected new or changed devices on smithi188 2024-03-24T02:55:10.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:09 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi188 to 8540M 2024-03-24T02:55:10.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:09 smithi188 ceph-mon[36206]: osdmap e39: 6 total, 6 up, 6 in 2024-03-24T02:55:10.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:09 smithi019 ceph-mon[35944]: Detected new or changed devices on smithi188 2024-03-24T02:55:10.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:09 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi188 to 8540M 2024-03-24T02:55:10.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:09 smithi019 ceph-mon[35944]: osdmap e39: 6 total, 6 up, 6 in 2024-03-24T02:55:10.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.497+0000 7f7d3f815700 1 -- 172.21.15.17:0/642651048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d381021c0 msgr2=0x7f7d381025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:55:10.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.497+0000 7f7d3f815700 1 --2- 172.21.15.17:0/642651048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d381021c0 0x7f7d381025e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d340099c0 tx=0x7f7d34009cd0 comp rx=0 tx=0).stop 2024-03-24T02:55:10.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.497+0000 7f7d3f815700 1 -- 172.21.15.17:0/642651048 shutdown_connections 2024-03-24T02:55:10.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.497+0000 7f7d3f815700 1 --2- 172.21.15.17:0/642651048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d38103d60 0x7f7d3810a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:10.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.497+0000 7f7d3f815700 1 --2- 172.21.15.17:0/642651048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d381021c0 0x7f7d381025e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:10.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.497+0000 7f7d3f815700 1 --2- 172.21.15.17:0/642651048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d381033c0 0x7f7d38103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:10.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.497+0000 7f7d3f815700 1 -- 172.21.15.17:0/642651048 >> 172.21.15.17:0/642651048 conn(0x7f7d380fd7a0 msgr2=0x7f7d380ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:55:10.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.497+0000 7f7d3f815700 1 -- 172.21.15.17:0/642651048 shutdown_connections 2024-03-24T02:55:10.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.497+0000 7f7d3f815700 1 -- 172.21.15.17:0/642651048 wait complete. 2024-03-24T02:55:10.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.498+0000 7f7d3f815700 1 Processor -- start 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.498+0000 7f7d3f815700 1 -- start start 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.498+0000 7f7d3f815700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d381021c0 0x7f7d38198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.498+0000 7f7d3f815700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d381033c0 0x7f7d38198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.498+0000 7f7d3f815700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d38103d60 0x7f7d3819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.498+0000 7f7d3f815700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3810de80 con 0x7f7d38103d60 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.498+0000 7f7d3f815700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3810dd00 con 0x7f7d381021c0 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.498+0000 7f7d3f815700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3810e000 con 0x7f7d381033c0 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.498+0000 7f7d3ddb2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d38103d60 0x7f7d3819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3ddb2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d38103d60 0x7f7d3819fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54034/0 (socket says 172.21.15.17:54034) 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3ddb2700 1 -- 172.21.15.17:0/1394541172 learned_addr learned my addr 172.21.15.17:0/1394541172 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3d5b1700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d381021c0 0x7f7d38198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3cdb0700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d381033c0 0x7f7d38198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3ddb2700 1 -- 172.21.15.17:0/1394541172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d381033c0 msgr2=0x7f7d38198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:55:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3ddb2700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d381033c0 0x7f7d38198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:10.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3ddb2700 1 -- 172.21.15.17:0/1394541172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d381021c0 msgr2=0x7f7d38198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:55:10.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3ddb2700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d381021c0 0x7f7d38198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:10.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3ddb2700 1 -- 172.21.15.17:0/1394541172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d381a01d0 con 0x7f7d38103d60 2024-03-24T02:55:10.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3cdb0700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d381033c0 0x7f7d38198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:55:10.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3ddb2700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d38103d60 0x7f7d3819fb30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000b9a0 tx=0x7f7d3000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:55:10.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d2e7fc700 1 -- 172.21.15.17:0/1394541172 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30019070 con 0x7f7d38103d60 2024-03-24T02:55:10.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d2e7fc700 1 -- 172.21.15.17:0/1394541172 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d3000ce10 con 0x7f7d38103d60 2024-03-24T02:55:10.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d2e7fc700 1 -- 172.21.15.17:0/1394541172 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d300073d0 con 0x7f7d38103d60 2024-03-24T02:55:10.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3f815700 1 -- 172.21.15.17:0/1394541172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d381a04c0 con 0x7f7d38103d60 2024-03-24T02:55:10.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.499+0000 7f7d3f815700 1 -- 172.21.15.17:0/1394541172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d381a0ad0 con 0x7f7d38103d60 2024-03-24T02:55:10.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.500+0000 7f7d3f815700 1 -- 172.21.15.17:0/1394541172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d38068e50 con 0x7f7d38103d60 2024-03-24T02:55:10.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.502+0000 7f7d2e7fc700 1 -- 172.21.15.17:0/1394541172 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7d3001c070 con 0x7f7d38103d60 2024-03-24T02:55:10.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.503+0000 7f7d2e7fc700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7d240a2910 0x7f7d240a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:55:10.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.503+0000 7f7d3d5b1700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7d240a2910 0x7f7d240a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:55:10.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.503+0000 7f7d2e7fc700 1 -- 172.21.15.17:0/1394541172 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f7d300cd400 con 0x7f7d38103d60 2024-03-24T02:55:10.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.503+0000 7f7d3d5b1700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7d240a2910 0x7f7d240a4dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400b580 tx=0x7f7d34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:55:10.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.507+0000 7f7d2e7fc700 1 -- 172.21.15.17:0/1394541172 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7d3009b2c0 con 0x7f7d38103d60 2024-03-24T02:55:10.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:10.722+0000 7f7d3f815700 1 -- 172.21.15.17:0/1394541172 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d380625d0 con 0x7f7d240a2910 2024-03-24T02:55:11.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:10 smithi188 ceph-mon[36206]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T02:55:11.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:10 smithi188 ceph-mon[36206]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:11.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:10 smithi188 ceph-mon[36206]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T02:55:11.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:10 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:55:11.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:10 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:55:11.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:10 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:11.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:10 smithi019 ceph-mon[35944]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T02:55:11.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:10 smithi019 ceph-mon[35944]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:11.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:10 smithi019 ceph-mon[35944]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T02:55:11.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:10 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:55:11.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:10 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:55:11.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:10 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:10 smithi017 ceph-mon[31926]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T02:55:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:10 smithi017 ceph-mon[31926]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:10 smithi017 ceph-mon[31926]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T02:55:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:10 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:55:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:10 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:55:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:10 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:11 smithi017 ceph-mon[31926]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:55:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:11 smithi017 ceph-mon[31926]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T02:55:12.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:11 smithi188 ceph-mon[36206]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:55:12.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:11 smithi188 ceph-mon[36206]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T02:55:12.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:11 smithi019 ceph-mon[35944]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:55:12.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:11 smithi019 ceph-mon[35944]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T02:55:12.955 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:12 smithi017 ceph-mon[31926]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-24T02:55:13.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:12 smithi188 ceph-mon[36206]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-24T02:55:13.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:12 smithi019 ceph-mon[35944]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-24T02:55:14.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:13 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/514328312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6713217d-57aa-4293-9cb7-de935d0a8537"}]: dispatch 2024-03-24T02:55:14.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:13 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/514328312' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6713217d-57aa-4293-9cb7-de935d0a8537"}]': finished 2024-03-24T02:55:14.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:13 smithi188 ceph-mon[36206]: osdmap e43: 7 total, 6 up, 7 in 2024-03-24T02:55:14.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:13 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:14.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:13 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/514328312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6713217d-57aa-4293-9cb7-de935d0a8537"}]: dispatch 2024-03-24T02:55:14.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:13 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/514328312' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6713217d-57aa-4293-9cb7-de935d0a8537"}]': finished 2024-03-24T02:55:14.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:13 smithi019 ceph-mon[35944]: osdmap e43: 7 total, 6 up, 7 in 2024-03-24T02:55:14.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:13 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:13 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/514328312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6713217d-57aa-4293-9cb7-de935d0a8537"}]: dispatch 2024-03-24T02:55:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:13 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/514328312' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6713217d-57aa-4293-9cb7-de935d0a8537"}]': finished 2024-03-24T02:55:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:13 smithi017 ceph-mon[31926]: osdmap e43: 7 total, 6 up, 7 in 2024-03-24T02:55:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:13 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:15.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:14 smithi188 ceph-mon[36206]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-03-24T02:55:15.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:14 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3736803723' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:55:15.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:14 smithi019 ceph-mon[35944]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-03-24T02:55:15.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:14 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3736803723' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:55:15.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:14 smithi017 ceph-mon[31926]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-03-24T02:55:15.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:14 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3736803723' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:55:16.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:15 smithi188 ceph-mon[36206]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T02:55:16.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:15 smithi019 ceph-mon[35944]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T02:55:16.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:15 smithi017 ceph-mon[31926]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T02:55:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:17 smithi017 ceph-mon[31926]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:18.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:17 smithi188 ceph-mon[36206]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:17 smithi019 ceph-mon[35944]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:19 smithi017 ceph-mon[31926]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:20.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:19 smithi188 ceph-mon[36206]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:20.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:19 smithi019 ceph-mon[35944]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:21.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:21 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T02:55:21.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:21 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:21.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:21 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T02:55:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:21 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:21 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T02:55:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:21 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:22.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:22 smithi017 ceph-mon[31926]: Deploying daemon osd.6 on smithi017 2024-03-24T02:55:22.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:22 smithi017 ceph-mon[31926]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:22.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:22 smithi188 ceph-mon[36206]: Deploying daemon osd.6 on smithi017 2024-03-24T02:55:22.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:22 smithi188 ceph-mon[36206]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:22.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:22 smithi019 ceph-mon[35944]: Deploying daemon osd.6 on smithi017 2024-03-24T02:55:22.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:22 smithi019 ceph-mon[35944]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:23 smithi019 ceph-mon[35944]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:24.219 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:23 smithi188 ceph-mon[36206]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:24.328 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:23 smithi017 ceph-mon[31926]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:26.133 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:25 smithi017 ceph-mon[31926]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:26.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:25 smithi188 ceph-mon[36206]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:26.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:25 smithi019 ceph-mon[35944]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:28.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:27 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:28.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:27 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:28.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:27 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:28.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:27 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:28.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:27 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:28.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:27 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:28.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:27 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:28.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:27 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:28.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:27 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:29.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:28 smithi188 ceph-mon[36206]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:29.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:28 smithi019 ceph-mon[35944]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:29.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:28 smithi017 ceph-mon[31926]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:30.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.781+0000 7f7d2e7fc700 1 -- 172.21.15.17:0/1394541172 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7d380625d0 con 0x7f7d240a2910 2024-03-24T02:55:30.781 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 6 on host 'smithi017' 2024-03-24T02:55:30.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.783+0000 7f7d3f815700 1 -- 172.21.15.17:0/1394541172 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7d240a2910 msgr2=0x7f7d240a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:55:30.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.783+0000 7f7d3f815700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7d240a2910 0x7f7d240a4dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400b580 tx=0x7f7d34005fb0 comp rx=0 tx=0).stop 2024-03-24T02:55:30.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.783+0000 7f7d3f815700 1 -- 172.21.15.17:0/1394541172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d38103d60 msgr2=0x7f7d3819fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:55:30.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.783+0000 7f7d3f815700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d38103d60 0x7f7d3819fb30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000b9a0 tx=0x7f7d3000bcb0 comp rx=0 tx=0).stop 2024-03-24T02:55:30.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.783+0000 7f7d3f815700 1 -- 172.21.15.17:0/1394541172 shutdown_connections 2024-03-24T02:55:30.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.783+0000 7f7d3f815700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7d240a2910 0x7f7d240a4dd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:30.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.783+0000 7f7d3f815700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d38103d60 0x7f7d3819fb30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:30.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.783+0000 7f7d3f815700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d381021c0 0x7f7d38198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:30.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.783+0000 7f7d3f815700 1 --2- 172.21.15.17:0/1394541172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d381033c0 0x7f7d38198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:30.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.783+0000 7f7d3f815700 1 -- 172.21.15.17:0/1394541172 >> 172.21.15.17:0/1394541172 conn(0x7f7d380fd7a0 msgr2=0x7f7d381067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:55:30.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.784+0000 7f7d3f815700 1 -- 172.21.15.17:0/1394541172 shutdown_connections 2024-03-24T02:55:30.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:55:30.784+0000 7f7d3f815700 1 -- 172.21.15.17:0/1394541172 wait complete. 2024-03-24T02:55:30.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:30.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:30.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:30.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:55:30.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:30.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:30 smithi017 ceph-mon[31926]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:30.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:30 smithi017 ceph-mon[31926]: from='osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T02:55:30.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:30.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:30.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:55:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:30 smithi188 ceph-mon[36206]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:30 smithi188 ceph-mon[36206]: from='osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T02:55:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:31.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:55:31.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:31.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:30 smithi019 ceph-mon[35944]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:31.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:30 smithi019 ceph-mon[35944]: from='osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T02:55:31.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:31.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:31.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:31.633 DEBUG:teuthology.orchestra.run.smithi017:osd.6> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.6.service 2024-03-24T02:55:31.635 INFO:tasks.cephadm:Deploying osd.7 on smithi019 with /dev/vg_nvme/lv_2... 2024-03-24T02:55:31.635 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-24T02:55:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T02:55:31.920 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:31 smithi017 ceph-mon[31926]: from='osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-24T02:55:31.920 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:31 smithi017 ceph-mon[31926]: osdmap e44: 7 total, 6 up, 7 in 2024-03-24T02:55:31.920 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:31 smithi017 ceph-mon[31926]: from='osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-03-24T02:55:31.920 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:31.920 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 02:55:31 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T02:55:31.791+0000 7f2efcf75700 -1 osd.6 0 waiting for initial osdmap 2024-03-24T02:55:31.921 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 02:55:31 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T02:55:31.801+0000 7f2ef8d68700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T02:55:32.020 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:31 smithi019 ceph-mon[35944]: from='osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-24T02:55:32.020 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:31 smithi019 ceph-mon[35944]: osdmap e44: 7 total, 6 up, 7 in 2024-03-24T02:55:32.020 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:31 smithi019 ceph-mon[35944]: from='osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-03-24T02:55:32.020 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:32.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:31 smithi188 ceph-mon[36206]: from='osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-24T02:55:32.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:31 smithi188 ceph-mon[36206]: osdmap e44: 7 total, 6 up, 7 in 2024-03-24T02:55:32.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:31 smithi188 ceph-mon[36206]: from='osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-03-24T02:55:32.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:33.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:32 smithi188 ceph-mon[36206]: from='osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-03-24T02:55:33.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:32 smithi188 ceph-mon[36206]: osdmap e45: 7 total, 6 up, 7 in 2024-03-24T02:55:33.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:33.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:33.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:32 smithi188 ceph-mon[36206]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:33.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:32 smithi019 ceph-mon[35944]: from='osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-03-24T02:55:33.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:32 smithi019 ceph-mon[35944]: osdmap e45: 7 total, 6 up, 7 in 2024-03-24T02:55:33.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:33.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:33.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:32 smithi019 ceph-mon[35944]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:33.105 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:32 smithi017 ceph-mon[31926]: from='osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-03-24T02:55:33.105 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:32 smithi017 ceph-mon[31926]: osdmap e45: 7 total, 6 up, 7 in 2024-03-24T02:55:33.106 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:33.106 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:33.106 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:32 smithi017 ceph-mon[31926]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-24T02:55:34.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:33 smithi188 ceph-mon[36206]: purged_snaps scrub starts 2024-03-24T02:55:34.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:33 smithi188 ceph-mon[36206]: purged_snaps scrub ok 2024-03-24T02:55:34.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:33 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:34.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:33 smithi188 ceph-mon[36206]: osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354] boot 2024-03-24T02:55:34.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:33 smithi188 ceph-mon[36206]: osdmap e46: 7 total, 7 up, 7 in 2024-03-24T02:55:34.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:33 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:34.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:33 smithi019 ceph-mon[35944]: purged_snaps scrub starts 2024-03-24T02:55:34.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:33 smithi019 ceph-mon[35944]: purged_snaps scrub ok 2024-03-24T02:55:34.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:33 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:34.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:33 smithi019 ceph-mon[35944]: osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354] boot 2024-03-24T02:55:34.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:33 smithi019 ceph-mon[35944]: osdmap e46: 7 total, 7 up, 7 in 2024-03-24T02:55:34.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:33 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:34.114 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:33 smithi017 ceph-mon[31926]: purged_snaps scrub starts 2024-03-24T02:55:34.114 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:33 smithi017 ceph-mon[31926]: purged_snaps scrub ok 2024-03-24T02:55:34.114 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:34.114 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:33 smithi017 ceph-mon[31926]: osd.6 [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354] boot 2024-03-24T02:55:34.114 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:33 smithi017 ceph-mon[31926]: osdmap e46: 7 total, 7 up, 7 in 2024-03-24T02:55:34.115 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T02:55:34.144 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-24T02:55:34.144 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-24T02:55:34.144 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-03-24T02:55:34.144 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-03-24T02:55:34.144 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112821 s, 929 MB/s 2024-03-24T02:55:34.144 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-03-24T02:55:34.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-03-24T02:55:35.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:34 smithi019 ceph-mon[35944]: osdmap e47: 7 total, 7 up, 7 in 2024-03-24T02:55:35.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:34 smithi019 ceph-mon[35944]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T02:55:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:34 smithi017 ceph-mon[31926]: osdmap e47: 7 total, 7 up, 7 in 2024-03-24T02:55:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:34 smithi017 ceph-mon[31926]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:35.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:34 smithi188 ceph-mon[36206]: osdmap e47: 7 total, 7 up, 7 in 2024-03-24T02:55:35.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:34 smithi188 ceph-mon[36206]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:36.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:36 smithi019 ceph-mon[35944]: Detected new or changed devices on smithi017 2024-03-24T02:55:36.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:36 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:36.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:36 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:36.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:36 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:36.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:36 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:36.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:36 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:36.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:36 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi017 to 5697M 2024-03-24T02:55:36.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:36 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:36.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:36 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:36.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:36 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:55:36.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:36 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:36.287 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:36 smithi019 ceph-mon[35944]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:36.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:36 smithi017 ceph-mon[31926]: Detected new or changed devices on smithi017 2024-03-24T02:55:36.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:36 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:36.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:36 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:36.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:36 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:36.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:36 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:36.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:36 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:36.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:36 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi017 to 5697M 2024-03-24T02:55:36.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:36 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:36.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:36 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:36.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:36 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:55:36.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:36 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:36.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:36 smithi017 ceph-mon[31926]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:36.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:36 smithi188 ceph-mon[36206]: Detected new or changed devices on smithi017 2024-03-24T02:55:36.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:36 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:36.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:36 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:36.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:36 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:36.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:36 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:36.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:36 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:55:36.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:36 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi017 to 5697M 2024-03-24T02:55:36.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:36 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:36.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:36 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:36.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:36 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:55:36.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:36 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:36.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:36 smithi188 ceph-mon[36206]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.817+0000 7f254c1eb700 1 -- 172.21.15.19:0/3453220691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2544107fa0 msgr2=0x7f254410a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:55:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.817+0000 7f254c1eb700 1 --2- 172.21.15.19:0/3453220691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2544107fa0 0x7f254410a390 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f253c00b0a0 tx=0x7f253c00b3b0 comp rx=0 tx=0).stop 2024-03-24T02:55:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.818+0000 7f254c1eb700 1 -- 172.21.15.19:0/3453220691 shutdown_connections 2024-03-24T02:55:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.818+0000 7f254c1eb700 1 --2- 172.21.15.19:0/3453220691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2544107fa0 0x7f254410a390 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.818+0000 7f254c1eb700 1 --2- 172.21.15.19:0/3453220691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2544102d40 0x7f2544105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.818+0000 7f254c1eb700 1 --2- 172.21.15.19:0/3453220691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544105670 0x7f2544107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.818+0000 7f254c1eb700 1 -- 172.21.15.19:0/3453220691 >> 172.21.15.19:0/3453220691 conn(0x7f25440fc690 msgr2=0x7f25440feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:55:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.818+0000 7f254c1eb700 1 -- 172.21.15.19:0/3453220691 shutdown_connections 2024-03-24T02:55:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.818+0000 7f254c1eb700 1 -- 172.21.15.19:0/3453220691 wait complete. 2024-03-24T02:55:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.818+0000 7f254c1eb700 1 Processor -- start 2024-03-24T02:55:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.818+0000 7f254c1eb700 1 -- start start 2024-03-24T02:55:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f254c1eb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2544102d40 0x7f2544198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:55:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f254c1eb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544105670 0x7f25441987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:55:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f254c1eb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2544107fa0 0x7f254419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f254c1eb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254410ddc0 con 0x7f2544105670 2024-03-24T02:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f254c1eb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254410dc40 con 0x7f2544102d40 2024-03-24T02:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f254c1eb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254410df40 con 0x7f2544107fa0 2024-03-24T02:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f2549f87700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2544102d40 0x7f2544198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f2549f87700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2544102d40 0x7f2544198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40092/0 (socket says 172.21.15.19:40092) 2024-03-24T02:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f2549f87700 1 -- 172.21.15.19:0/698802349 learned_addr learned my addr 172.21.15.19:0/698802349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-24T02:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f254a788700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2544107fa0 0x7f254419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f2549786700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544105670 0x7f25441987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f2549f87700 1 -- 172.21.15.19:0/698802349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2544107fa0 msgr2=0x7f254419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f2549f87700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2544107fa0 0x7f254419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f2549f87700 1 -- 172.21.15.19:0/698802349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544105670 msgr2=0x7f25441987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f2549f87700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544105670 0x7f25441987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f2549f87700 1 -- 172.21.15.19:0/698802349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254419fee0 con 0x7f2544102d40 2024-03-24T02:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f2549786700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544105670 0x7f25441987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f2549f87700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2544102d40 0x7f2544198280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f254000c9b0 tx=0x7f254000cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f253affd700 1 -- 172.21.15.19:0/698802349 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540007b00 con 0x7f2544102d40 2024-03-24T02:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f253affd700 1 -- 172.21.15.19:0/698802349 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2540007ca0 con 0x7f2544102d40 2024-03-24T02:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f254c1eb700 1 -- 172.21.15.19:0/698802349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25441a01d0 con 0x7f2544102d40 2024-03-24T02:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.819+0000 7f253affd700 1 -- 172.21.15.19:0/698802349 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540018820 con 0x7f2544102d40 2024-03-24T02:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.820+0000 7f254c1eb700 1 -- 172.21.15.19:0/698802349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25441a0710 con 0x7f2544102d40 2024-03-24T02:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.820+0000 7f254c1eb700 1 -- 172.21.15.19:0/698802349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2544068e50 con 0x7f2544102d40 2024-03-24T02:55:36.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.822+0000 7f253affd700 1 -- 172.21.15.19:0/698802349 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f254000fa30 con 0x7f2544102d40 2024-03-24T02:55:36.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.822+0000 7f253affd700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25300a2890 0x7f25300a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:55:36.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.822+0000 7f253affd700 1 -- 172.21.15.19:0/698802349 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f25400cddc0 con 0x7f2544102d40 2024-03-24T02:55:36.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.822+0000 7f2549786700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25300a2890 0x7f25300a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:55:36.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.823+0000 7f2549786700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25300a2890 0x7f25300a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2534005d90 tx=0x7f2534005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:55:36.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:36.827+0000 7f253affd700 1 -- 172.21.15.19:0/698802349 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2540014030 con 0x7f2544102d40 2024-03-24T02:55:37.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:37.041+0000 7f254c1eb700 1 -- 172.21.15.19:0/698802349 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f25440625d0 con 0x7f25300a2890 2024-03-24T02:55:37.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:37 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:55:37.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:37 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:55:37.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:37 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:37.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:37 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:55:37.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:37 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:55:37.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:37 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:37.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:37 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:55:37.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:37 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:55:37.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:37 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:38 smithi019 ceph-mon[35944]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:55:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:38 smithi019 ceph-mon[35944]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:38 smithi017 ceph-mon[31926]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:55:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:38 smithi017 ceph-mon[31926]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:38.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:38 smithi188 ceph-mon[36206]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:55:38.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:38 smithi188 ceph-mon[36206]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:40.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:39 smithi188 ceph-mon[36206]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:39 smithi019 ceph-mon[35944]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:40.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:39 smithi017 ceph-mon[31926]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:41.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:40 smithi188 ceph-mon[36206]: from='client.? 172.21.15.19:0/2914245037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aa6635c-d640-46d9-9124-3e7afe24e65d"}]: dispatch 2024-03-24T02:55:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:40 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aa6635c-d640-46d9-9124-3e7afe24e65d"}]: dispatch 2024-03-24T02:55:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:40 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4aa6635c-d640-46d9-9124-3e7afe24e65d"}]': finished 2024-03-24T02:55:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:40 smithi188 ceph-mon[36206]: osdmap e48: 8 total, 7 up, 8 in 2024-03-24T02:55:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:40 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:55:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:40 smithi188 ceph-mon[36206]: from='client.? 172.21.15.19:0/225803432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:55:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:40 smithi019 ceph-mon[35944]: from='client.? 172.21.15.19:0/2914245037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aa6635c-d640-46d9-9124-3e7afe24e65d"}]: dispatch 2024-03-24T02:55:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:40 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aa6635c-d640-46d9-9124-3e7afe24e65d"}]: dispatch 2024-03-24T02:55:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:40 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4aa6635c-d640-46d9-9124-3e7afe24e65d"}]': finished 2024-03-24T02:55:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:40 smithi019 ceph-mon[35944]: osdmap e48: 8 total, 7 up, 8 in 2024-03-24T02:55:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:40 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:55:41.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:40 smithi019 ceph-mon[35944]: from='client.? 172.21.15.19:0/225803432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:55:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:40 smithi017 ceph-mon[31926]: from='client.? 172.21.15.19:0/2914245037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aa6635c-d640-46d9-9124-3e7afe24e65d"}]: dispatch 2024-03-24T02:55:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:40 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aa6635c-d640-46d9-9124-3e7afe24e65d"}]: dispatch 2024-03-24T02:55:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:40 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4aa6635c-d640-46d9-9124-3e7afe24e65d"}]': finished 2024-03-24T02:55:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:40 smithi017 ceph-mon[31926]: osdmap e48: 8 total, 7 up, 8 in 2024-03-24T02:55:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:40 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:55:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:40 smithi017 ceph-mon[31926]: from='client.? 172.21.15.19:0/225803432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:55:42.222 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:41 smithi019 ceph-mon[35944]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:42.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:41 smithi188 ceph-mon[36206]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:42.375 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:41 smithi017 ceph-mon[31926]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:45.178 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:44 smithi019 ceph-mon[35944]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:45.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:44 smithi188 ceph-mon[36206]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:45.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:44 smithi017 ceph-mon[31926]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:47.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:46 smithi188 ceph-mon[36206]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:47.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:46 smithi019 ceph-mon[35944]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:47.455 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:46 smithi017 ceph-mon[31926]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:48.105 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:47 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T02:55:48.105 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:47 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:48.105 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:47 smithi019 ceph-mon[35944]: Deploying daemon osd.7 on smithi019 2024-03-24T02:55:48.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:47 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T02:55:48.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:47 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:48.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:47 smithi188 ceph-mon[36206]: Deploying daemon osd.7 on smithi019 2024-03-24T02:55:48.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:47 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T02:55:48.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:47 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:55:48.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:47 smithi017 ceph-mon[31926]: Deploying daemon osd.7 on smithi019 2024-03-24T02:55:49.241 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:48 smithi019 ceph-mon[35944]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:49.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:48 smithi188 ceph-mon[36206]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:49.455 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:48 smithi017 ceph-mon[31926]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:51.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:50 smithi019 ceph-mon[35944]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:51.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:50 smithi188 ceph-mon[36206]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:51.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:50 smithi017 ceph-mon[31926]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:53.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:52 smithi019 ceph-mon[35944]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:53.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:52 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:53.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:52 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:53.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:52 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:53.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:52 smithi188 ceph-mon[36206]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:53.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:52 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:53.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:52 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:53.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:52 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:52 smithi017 ceph-mon[31926]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:52 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:52 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:52 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:55:55.056 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:54 smithi019 ceph-mon[35944]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:55.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:54 smithi188 ceph-mon[36206]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:55.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:54 smithi017 ceph-mon[31926]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.359+0000 7f253affd700 1 -- 172.21.15.19:0/698802349 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f25440625d0 con 0x7f25300a2890 2024-03-24T02:55:56.360 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 7 on host 'smithi019' 2024-03-24T02:55:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.362+0000 7f254c1eb700 1 -- 172.21.15.19:0/698802349 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25300a2890 msgr2=0x7f25300a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:55:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.362+0000 7f254c1eb700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25300a2890 0x7f25300a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2534005d90 tx=0x7f2534005d00 comp rx=0 tx=0).stop 2024-03-24T02:55:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.362+0000 7f254c1eb700 1 -- 172.21.15.19:0/698802349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2544102d40 msgr2=0x7f2544198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:55:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.362+0000 7f254c1eb700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2544102d40 0x7f2544198280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f254000c9b0 tx=0x7f254000cd70 comp rx=0 tx=0).stop 2024-03-24T02:55:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.362+0000 7f254c1eb700 1 -- 172.21.15.19:0/698802349 shutdown_connections 2024-03-24T02:55:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.362+0000 7f254c1eb700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25300a2890 0x7f25300a4d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.362+0000 7f254c1eb700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2544107fa0 0x7f254419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.362+0000 7f254c1eb700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2544102d40 0x7f2544198280 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.362+0000 7f254c1eb700 1 --2- 172.21.15.19:0/698802349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544105670 0x7f25441987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:55:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.362+0000 7f254c1eb700 1 -- 172.21.15.19:0/698802349 >> 172.21.15.19:0/698802349 conn(0x7f25440fc690 msgr2=0x7f2544103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:55:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.362+0000 7f254c1eb700 1 -- 172.21.15.19:0/698802349 shutdown_connections 2024-03-24T02:55:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:55:56.362+0000 7f254c1eb700 1 -- 172.21.15.19:0/698802349 wait complete. 2024-03-24T02:55:56.920 DEBUG:teuthology.orchestra.run.smithi019:osd.7> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.7.service 2024-03-24T02:55:56.923 INFO:tasks.cephadm:Deploying osd.8 on smithi188 with /dev/vg_nvme/lv_2... 2024-03-24T02:55:56.923 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-24T02:55:57.169 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T02:55:57.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:57 smithi188 ceph-mon[36206]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:57.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:57 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:57.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:57 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:57.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:57 smithi188 ceph-mon[36206]: from='osd.7 [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T02:55:57.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:57 smithi188 ceph-mon[36206]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T02:55:57.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:57 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:57.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:57 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:57.659 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:57 smithi019 ceph-mon[35944]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:57.659 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:57 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:57.659 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:57 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:57.659 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:57 smithi019 ceph-mon[35944]: from='osd.7 [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T02:55:57.660 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:57 smithi019 ceph-mon[35944]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T02:55:57.660 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:57 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:57.660 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:57 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:57.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:57 smithi017 ceph-mon[31926]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:57.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:57 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:57.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:57 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:57.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:57 smithi017 ceph-mon[31926]: from='osd.7 [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T02:55:57.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:57 smithi017 ceph-mon[31926]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T02:55:57.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:57 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:57.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:57 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:55:58.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:58 smithi188 ceph-mon[36206]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-24T02:55:58.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:58 smithi188 ceph-mon[36206]: osdmap e49: 8 total, 7 up, 8 in 2024-03-24T02:55:58.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:58 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:55:58.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:58 smithi188 ceph-mon[36206]: from='osd.7 [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:55:58.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:58 smithi188 ceph-mon[36206]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:55:58.663 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:58 smithi019 ceph-mon[35944]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-24T02:55:58.663 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:58 smithi019 ceph-mon[35944]: osdmap e49: 8 total, 7 up, 8 in 2024-03-24T02:55:58.663 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:58 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:55:58.663 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:58 smithi019 ceph-mon[35944]: from='osd.7 [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:55:58.663 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:58 smithi019 ceph-mon[35944]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:55:58.663 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 02:55:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T02:55:58.344+0000 7fcc4deac700 -1 osd.7 0 waiting for initial osdmap 2024-03-24T02:55:58.663 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 02:55:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T02:55:58.353+0000 7fcc49c9f700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T02:55:58.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:58 smithi017 ceph-mon[31926]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-24T02:55:58.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:58 smithi017 ceph-mon[31926]: osdmap e49: 8 total, 7 up, 8 in 2024-03-24T02:55:58.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:58 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:55:58.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:58 smithi017 ceph-mon[31926]: from='osd.7 [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:55:58.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:58 smithi017 ceph-mon[31926]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:55:59.240 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-24T02:55:59.240 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-24T02:55:59.240 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-03-24T02:55:59.240 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-03-24T02:55:59.240 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114951 s, 912 MB/s 2024-03-24T02:55:59.240 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-03-24T02:55:59.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:59 smithi188 ceph-mon[36206]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:59.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:59 smithi188 ceph-mon[36206]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-24T02:55:59.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:59 smithi188 ceph-mon[36206]: osdmap e50: 8 total, 7 up, 8 in 2024-03-24T02:55:59.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:59 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:55:59.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:55:59 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:55:59.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:59 smithi017 ceph-mon[31926]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:59.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:59 smithi017 ceph-mon[31926]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-24T02:55:59.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:59 smithi017 ceph-mon[31926]: osdmap e50: 8 total, 7 up, 8 in 2024-03-24T02:55:59.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:55:59.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:55:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:55:59.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:59 smithi019 ceph-mon[35944]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-24T02:55:59.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:59 smithi019 ceph-mon[35944]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-24T02:55:59.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:59 smithi019 ceph-mon[35944]: osdmap e50: 8 total, 7 up, 8 in 2024-03-24T02:55:59.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:59 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:55:59.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:55:59 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:55:59.839 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi188:vg_nvme/lv_2 2024-03-24T02:56:00.082 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T02:56:00.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:00 smithi188 ceph-mon[36206]: purged_snaps scrub starts 2024-03-24T02:56:00.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:00 smithi188 ceph-mon[36206]: purged_snaps scrub ok 2024-03-24T02:56:00.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:00 smithi188 ceph-mon[36206]: osd.7 [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507] boot 2024-03-24T02:56:00.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:00 smithi188 ceph-mon[36206]: osdmap e51: 8 total, 8 up, 8 in 2024-03-24T02:56:00.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:00 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:56:00.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:00 smithi188 ceph-mon[36206]: osdmap e52: 8 total, 8 up, 8 in 2024-03-24T02:56:00.566 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:00 smithi019 ceph-mon[35944]: purged_snaps scrub starts 2024-03-24T02:56:00.566 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:00 smithi019 ceph-mon[35944]: purged_snaps scrub ok 2024-03-24T02:56:00.566 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:00 smithi019 ceph-mon[35944]: osd.7 [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507] boot 2024-03-24T02:56:00.566 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:00 smithi019 ceph-mon[35944]: osdmap e51: 8 total, 8 up, 8 in 2024-03-24T02:56:00.566 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:00 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:56:00.566 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:00 smithi019 ceph-mon[35944]: osdmap e52: 8 total, 8 up, 8 in 2024-03-24T02:56:00.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:00 smithi017 ceph-mon[31926]: purged_snaps scrub starts 2024-03-24T02:56:00.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:00 smithi017 ceph-mon[31926]: purged_snaps scrub ok 2024-03-24T02:56:00.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:00 smithi017 ceph-mon[31926]: osd.7 [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507] boot 2024-03-24T02:56:00.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:00 smithi017 ceph-mon[31926]: osdmap e51: 8 total, 8 up, 8 in 2024-03-24T02:56:00.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:00 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T02:56:00.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:00 smithi017 ceph-mon[31926]: osdmap e52: 8 total, 8 up, 8 in 2024-03-24T02:56:01.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.843+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/1469423931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e14103e50 msgr2=0x7f6e1410a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:01.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.843+0000 7f6e1b0c7700 1 --2- 172.21.15.188:0/1469423931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e14103e50 0x7f6e1410a700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009f90 tx=0x7f6e0c00b330 comp rx=0 tx=0).stop 2024-03-24T02:56:01.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/1469423931 shutdown_connections 2024-03-24T02:56:01.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 --2- 172.21.15.188:0/1469423931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e14103e50 0x7f6e1410a700 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:01.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 --2- 172.21.15.188:0/1469423931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e141004d0 0x7f6e141008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:01.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 --2- 172.21.15.188:0/1469423931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e141034b0 0x7f6e14103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:01.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/1469423931 >> 172.21.15.188:0/1469423931 conn(0x7f6e140fbe30 msgr2=0x7f6e140fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:56:01.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/1469423931 shutdown_connections 2024-03-24T02:56:01.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/1469423931 wait complete. 2024-03-24T02:56:01.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 Processor -- start 2024-03-24T02:56:01.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 -- start start 2024-03-24T02:56:01.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e141004d0 0x7f6e14198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:56:01.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e141034b0 0x7f6e14198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:56:01.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e14103e50 0x7f6e1419f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1410de00 con 0x7f6e141004d0 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1410dc80 con 0x7f6e141034b0 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e1b0c7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1410df80 con 0x7f6e14103e50 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e19664700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e14103e50 0x7f6e1419f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e19664700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e14103e50 0x7f6e1419f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:58646/0 (socket says 172.21.15.188:58646) 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e18e63700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e141004d0 0x7f6e14198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.844+0000 7f6e19664700 1 -- 172.21.15.188:0/2675491391 learned_addr learned my addr 172.21.15.188:0/2675491391 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e13fff700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e141034b0 0x7f6e14198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e19664700 1 -- 172.21.15.188:0/2675491391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e141034b0 msgr2=0x7f6e14198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e19664700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e141034b0 0x7f6e14198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e19664700 1 -- 172.21.15.188:0/2675491391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e141004d0 msgr2=0x7f6e14198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e19664700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e141004d0 0x7f6e14198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:01.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e19664700 1 -- 172.21.15.188:0/2675491391 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1419ffb0 con 0x7f6e14103e50 2024-03-24T02:56:01.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e18e63700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e141004d0 0x7f6e14198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:56:01.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e19664700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e14103e50 0x7f6e1419f910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c007cd0 tx=0x7f6e0c007d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:56:01.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e11ffb700 1 -- 172.21.15.188:0/2675491391 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c01f070 con 0x7f6e14103e50 2024-03-24T02:56:01.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/2675491391 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e141a0240 con 0x7f6e14103e50 2024-03-24T02:56:01.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e11ffb700 1 -- 172.21.15.188:0/2675491391 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e0c005530 con 0x7f6e14103e50 2024-03-24T02:56:01.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e11ffb700 1 -- 172.21.15.188:0/2675491391 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c01b640 con 0x7f6e14103e50 2024-03-24T02:56:01.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.845+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/2675491391 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e141a06f0 con 0x7f6e14103e50 2024-03-24T02:56:01.849 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.846+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/2675491391 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e14068e50 con 0x7f6e14103e50 2024-03-24T02:56:01.849 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.847+0000 7f6e11ffb700 1 -- 172.21.15.188:0/2675491391 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6e0c022070 con 0x7f6e14103e50 2024-03-24T02:56:01.849 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.848+0000 7f6e11ffb700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6e040a2890 0x7f6e040a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:56:01.849 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.848+0000 7f6e11ffb700 1 -- 172.21.15.188:0/2675491391 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f6e0c0ceb30 con 0x7f6e14103e50 2024-03-24T02:56:01.849 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.848+0000 7f6e18e63700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6e040a2890 0x7f6e040a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:56:01.849 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.848+0000 7f6e18e63700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6e040a2890 0x7f6e040a4d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6e1406a6b0 tx=0x7f6e08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:56:01.851 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:01.851+0000 7f6e11ffb700 1 -- 172.21.15.188:0/2675491391 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6e0c09c7a0 con 0x7f6e14103e50 2024-03-24T02:56:01.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:01.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: Detected new or changed devices on smithi019 2024-03-24T02:56:01.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:01.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:01.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:01.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:01.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:01.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi019 to 5694M 2024-03-24T02:56:01.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:01.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:01.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:01.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:01.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:01.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:01.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:01.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:01.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:01 smithi017 ceph-mon[31926]: osdmap e53: 8 total, 8 up, 8 in 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: Detected new or changed devices on smithi019 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi019 to 5694M 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:02.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:02.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:02.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:02.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:01 smithi188 ceph-mon[36206]: osdmap e53: 8 total, 8 up, 8 in 2024-03-24T02:56:02.058 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:02.058+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/2675491391 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e140625d0 con 0x7f6e040a2890 2024-03-24T02:56:02.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: Detected new or changed devices on smithi019 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi019 to 5694M 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:02.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:02.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:02.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:01 smithi019 ceph-mon[35944]: osdmap e53: 8 total, 8 up, 8 in 2024-03-24T02:56:02.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:02 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:56:02.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:02 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:56:02.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:02 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:02.960 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:02 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:56:02.960 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:02 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:56:02.960 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:02 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:03.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:02 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:56:03.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:02 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:56:03.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:02 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:03.896 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:03 smithi188 ceph-mon[36206]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:03.896 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:03 smithi188 ceph-mon[36206]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:56:03.897 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:03 smithi188 ceph-mon[36206]: osdmap e54: 8 total, 8 up, 8 in 2024-03-24T02:56:03.955 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:03 smithi017 ceph-mon[31926]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:03.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:03 smithi017 ceph-mon[31926]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:56:03.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:03 smithi017 ceph-mon[31926]: osdmap e54: 8 total, 8 up, 8 in 2024-03-24T02:56:04.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:03 smithi019 ceph-mon[35944]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:04.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:03 smithi019 ceph-mon[35944]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:56:04.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:03 smithi019 ceph-mon[35944]: osdmap e54: 8 total, 8 up, 8 in 2024-03-24T02:56:04.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:04 smithi017 ceph-mon[31926]: osdmap e55: 8 total, 8 up, 8 in 2024-03-24T02:56:05.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:04 smithi188 ceph-mon[36206]: osdmap e55: 8 total, 8 up, 8 in 2024-03-24T02:56:05.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:04 smithi019 ceph-mon[35944]: osdmap e55: 8 total, 8 up, 8 in 2024-03-24T02:56:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:05 smithi017 ceph-mon[31926]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:05 smithi017 ceph-mon[31926]: from='client.? 172.21.15.188:0/810734769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9df9a933-c0be-4277-9837-95fb58c813d6"}]: dispatch 2024-03-24T02:56:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:05 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9df9a933-c0be-4277-9837-95fb58c813d6"}]: dispatch 2024-03-24T02:56:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:05 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9df9a933-c0be-4277-9837-95fb58c813d6"}]': finished 2024-03-24T02:56:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:05 smithi017 ceph-mon[31926]: osdmap e56: 9 total, 8 up, 9 in 2024-03-24T02:56:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:05 smithi017 ceph-mon[31926]: from='client.? 172.21.15.188:0/827944173' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:56:06.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:05 smithi188 ceph-mon[36206]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:06.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:05 smithi188 ceph-mon[36206]: from='client.? 172.21.15.188:0/810734769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9df9a933-c0be-4277-9837-95fb58c813d6"}]: dispatch 2024-03-24T02:56:06.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:05 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9df9a933-c0be-4277-9837-95fb58c813d6"}]: dispatch 2024-03-24T02:56:06.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:05 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9df9a933-c0be-4277-9837-95fb58c813d6"}]': finished 2024-03-24T02:56:06.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:05 smithi188 ceph-mon[36206]: osdmap e56: 9 total, 8 up, 9 in 2024-03-24T02:56:06.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:06.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:05 smithi188 ceph-mon[36206]: from='client.? 172.21.15.188:0/827944173' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:56:06.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:05 smithi019 ceph-mon[35944]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:06.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:05 smithi019 ceph-mon[35944]: from='client.? 172.21.15.188:0/810734769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9df9a933-c0be-4277-9837-95fb58c813d6"}]: dispatch 2024-03-24T02:56:06.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:05 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9df9a933-c0be-4277-9837-95fb58c813d6"}]: dispatch 2024-03-24T02:56:06.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:05 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9df9a933-c0be-4277-9837-95fb58c813d6"}]': finished 2024-03-24T02:56:06.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:05 smithi019 ceph-mon[35944]: osdmap e56: 9 total, 8 up, 9 in 2024-03-24T02:56:06.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:06.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:05 smithi019 ceph-mon[35944]: from='client.? 172.21.15.188:0/827944173' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:56:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:07 smithi017 ceph-mon[31926]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:08.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:07 smithi188 ceph-mon[36206]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:08.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:07 smithi019 ceph-mon[35944]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:09.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:09 smithi017 ceph-mon[31926]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:09.972 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:09 smithi188 ceph-mon[36206]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:10.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:09 smithi019 ceph-mon[35944]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:11 smithi017 ceph-mon[31926]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:11.964 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:11 smithi188 ceph-mon[36206]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:12.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:11 smithi019 ceph-mon[35944]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:12.925 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:12 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-24T02:56:12.925 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:12 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:12.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:12 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-24T02:56:12.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:12 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:12.971 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:12 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-24T02:56:12.972 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:12 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:13.865 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:13 smithi188 ceph-mon[36206]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:13.865 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:13 smithi188 ceph-mon[36206]: Deploying daemon osd.8 on smithi188 2024-03-24T02:56:13.955 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:13 smithi017 ceph-mon[31926]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:13.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:13 smithi017 ceph-mon[31926]: Deploying daemon osd.8 on smithi188 2024-03-24T02:56:14.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:13 smithi019 ceph-mon[35944]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:14.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:13 smithi019 ceph-mon[35944]: Deploying daemon osd.8 on smithi188 2024-03-24T02:56:15.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:15 smithi017 ceph-mon[31926]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:15.967 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:15 smithi188 ceph-mon[36206]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:16.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:15 smithi019 ceph-mon[35944]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:17.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:17 smithi017 ceph-mon[31926]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:18.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:17 smithi188 ceph-mon[36206]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:18.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:17 smithi019 ceph-mon[35944]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:19.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:18 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:19.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:18 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:19.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:18 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:19.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:18 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:19.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:18 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:19.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:18 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:19.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:18 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:19.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:18 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:19.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:18 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:20.035 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:19 smithi188 ceph-mon[36206]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:20.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:19 smithi019 ceph-mon[35944]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:19 smithi017 ceph-mon[31926]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:21.325 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.326+0000 7f6e11ffb700 1 -- 172.21.15.188:0/2675491391 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6e140625d0 con 0x7f6e040a2890 2024-03-24T02:56:21.325 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 8 on host 'smithi188' 2024-03-24T02:56:21.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.328+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/2675491391 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6e040a2890 msgr2=0x7f6e040a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:21.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.328+0000 7f6e1b0c7700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6e040a2890 0x7f6e040a4d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6e1406a6b0 tx=0x7f6e08009260 comp rx=0 tx=0).stop 2024-03-24T02:56:21.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.328+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/2675491391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e14103e50 msgr2=0x7f6e1419f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:21.328 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.328+0000 7f6e1b0c7700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e14103e50 0x7f6e1419f910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c007cd0 tx=0x7f6e0c007d00 comp rx=0 tx=0).stop 2024-03-24T02:56:21.328 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.328+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/2675491391 shutdown_connections 2024-03-24T02:56:21.328 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.328+0000 7f6e1b0c7700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6e040a2890 0x7f6e040a4d50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:21.328 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.328+0000 7f6e1b0c7700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e14103e50 0x7f6e1419f910 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:21.328 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.328+0000 7f6e1b0c7700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e141004d0 0x7f6e14198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:21.328 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.328+0000 7f6e1b0c7700 1 --2- 172.21.15.188:0/2675491391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e141034b0 0x7f6e14198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:21.328 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.328+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/2675491391 >> 172.21.15.188:0/2675491391 conn(0x7f6e140fbe30 msgr2=0x7f6e14106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:56:21.328 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.328+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/2675491391 shutdown_connections 2024-03-24T02:56:21.328 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:56:21.328+0000 7f6e1b0c7700 1 -- 172.21.15.188:0/2675491391 wait complete. 2024-03-24T02:56:21.924 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:21 smithi188 ceph-mon[36206]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:21.924 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:21 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:21.924 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:21 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:21.925 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:21 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:21.925 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:21 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:21.925 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:21 smithi188 ceph-mon[36206]: from='osd.8 [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-24T02:56:21.925 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:21 smithi188 ceph-mon[36206]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-24T02:56:21.952 DEBUG:teuthology.orchestra.run.smithi188:osd.8> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.8.service 2024-03-24T02:56:21.954 INFO:tasks.cephadm:Deploying osd.9 on smithi017 with /dev/vg_nvme/lv_1... 2024-03-24T02:56:21.955 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-24T02:56:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:21 smithi019 ceph-mon[35944]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:21 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:21 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:21 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:21 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:21 smithi019 ceph-mon[35944]: from='osd.8 [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-24T02:56:22.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:21 smithi019 ceph-mon[35944]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-24T02:56:22.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:21 smithi017 ceph-mon[31926]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:21 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:21 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:21 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:21 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:21 smithi017 ceph-mon[31926]: from='osd.8 [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-24T02:56:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:21 smithi017 ceph-mon[31926]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-24T02:56:22.207 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:56:23.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:23 smithi188 ceph-mon[36206]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:23.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:23 smithi188 ceph-mon[36206]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-24T02:56:23.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:23 smithi188 ceph-mon[36206]: osdmap e57: 9 total, 8 up, 9 in 2024-03-24T02:56:23.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:23 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:23.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:23 smithi188 ceph-mon[36206]: from='osd.8 [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:56:23.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:23 smithi188 ceph-mon[36206]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:56:23.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:23 smithi017 ceph-mon[31926]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:23.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:23 smithi017 ceph-mon[31926]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-24T02:56:23.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:23 smithi017 ceph-mon[31926]: osdmap e57: 9 total, 8 up, 9 in 2024-03-24T02:56:23.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:23 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:23.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:23 smithi017 ceph-mon[31926]: from='osd.8 [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:56:23.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:23 smithi017 ceph-mon[31926]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:56:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:23 smithi019 ceph-mon[35944]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:23 smithi019 ceph-mon[35944]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-24T02:56:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:23 smithi019 ceph-mon[35944]: osdmap e57: 9 total, 8 up, 9 in 2024-03-24T02:56:24.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:23 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:24.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:23 smithi019 ceph-mon[35944]: from='osd.8 [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:56:24.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:23 smithi019 ceph-mon[35944]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:56:24.388 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-24T02:56:24.388 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-24T02:56:24.388 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2024-03-24T02:56:24.388 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2024-03-24T02:56:24.389 INFO:teuthology.orchestra.run.smithi017.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00870472 s, 1.2 GB/s 2024-03-24T02:56:24.389 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2024-03-24T02:56:24.650 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 02:56:24 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T02:56:24.375+0000 7fd6e0af1700 -1 osd.8 0 waiting for initial osdmap 2024-03-24T02:56:24.650 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 02:56:24 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T02:56:24.389+0000 7fd6d88dc700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T02:56:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:24 smithi017 ceph-mon[31926]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-03-24T02:56:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:24 smithi017 ceph-mon[31926]: osdmap e58: 9 total, 8 up, 9 in 2024-03-24T02:56:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:24 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:24 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:24 smithi017 ceph-mon[31926]: from='osd.8 ' entity='osd.8' 2024-03-24T02:56:25.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:24 smithi188 ceph-mon[36206]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-03-24T02:56:25.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:24 smithi188 ceph-mon[36206]: osdmap e58: 9 total, 8 up, 9 in 2024-03-24T02:56:25.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:24 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:25.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:24 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:25.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:24 smithi188 ceph-mon[36206]: from='osd.8 ' entity='osd.8' 2024-03-24T02:56:25.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:24 smithi019 ceph-mon[35944]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-03-24T02:56:25.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:24 smithi019 ceph-mon[35944]: osdmap e58: 9 total, 8 up, 9 in 2024-03-24T02:56:25.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:24 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:25.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:24 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:25.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:24 smithi019 ceph-mon[35944]: from='osd.8 ' entity='osd.8' 2024-03-24T02:56:25.212 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi017:vg_nvme/lv_1 2024-03-24T02:56:25.469 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:56:25.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: purged_snaps scrub starts 2024-03-24T02:56:25.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: purged_snaps scrub ok 2024-03-24T02:56:25.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:25.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:25.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: osd.8 [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279] boot 2024-03-24T02:56:25.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: osdmap e59: 9 total, 9 up, 9 in 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: osdmap e60: 9 total, 9 up, 9 in 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:25.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:25 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:26.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: purged_snaps scrub starts 2024-03-24T02:56:26.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: purged_snaps scrub ok 2024-03-24T02:56:26.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:26.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:26.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: osd.8 [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279] boot 2024-03-24T02:56:26.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: osdmap e59: 9 total, 9 up, 9 in 2024-03-24T02:56:26.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: osdmap e60: 9 total, 9 up, 9 in 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:26.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:25 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:26.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: purged_snaps scrub starts 2024-03-24T02:56:26.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: purged_snaps scrub ok 2024-03-24T02:56:26.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-03-24T02:56:26.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: osd.8 [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279] boot 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: osdmap e59: 9 total, 9 up, 9 in 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: osdmap e60: 9 total, 9 up, 9 in 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:26.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:26.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:26.072 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:25 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:26.893 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:26 smithi017 ceph-mon[31926]: Detected new or changed devices on smithi188 2024-03-24T02:56:26.893 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:26 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi188 to 5693M 2024-03-24T02:56:26.893 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:26 smithi017 ceph-mon[31926]: osdmap e61: 9 total, 9 up, 9 in 2024-03-24T02:56:27.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:26 smithi188 ceph-mon[36206]: Detected new or changed devices on smithi188 2024-03-24T02:56:27.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:26 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi188 to 5693M 2024-03-24T02:56:27.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:26 smithi188 ceph-mon[36206]: osdmap e61: 9 total, 9 up, 9 in 2024-03-24T02:56:27.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:26 smithi019 ceph-mon[35944]: Detected new or changed devices on smithi188 2024-03-24T02:56:27.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:26 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi188 to 5693M 2024-03-24T02:56:27.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:26 smithi019 ceph-mon[35944]: osdmap e61: 9 total, 9 up, 9 in 2024-03-24T02:56:27.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.133+0000 7f97a2135700 1 -- 172.21.15.17:0/3521129035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f979c1021c0 msgr2=0x7f979c1025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.133+0000 7f97a2135700 1 --2- 172.21.15.17:0/3521129035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f979c1021c0 0x7f979c1025e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f978c0099c0 tx=0x7f978c009cd0 comp rx=0 tx=0).stop 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.133+0000 7f97a2135700 1 -- 172.21.15.17:0/3521129035 shutdown_connections 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.133+0000 7f97a2135700 1 --2- 172.21.15.17:0/3521129035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f979c103d60 0x7f979c10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.133+0000 7f97a2135700 1 --2- 172.21.15.17:0/3521129035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f979c1021c0 0x7f979c1025e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.133+0000 7f97a2135700 1 --2- 172.21.15.17:0/3521129035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f979c1033c0 0x7f979c103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.133+0000 7f97a2135700 1 -- 172.21.15.17:0/3521129035 >> 172.21.15.17:0/3521129035 conn(0x7f979c0fd7a0 msgr2=0x7f979c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.133+0000 7f97a2135700 1 -- 172.21.15.17:0/3521129035 shutdown_connections 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.133+0000 7f97a2135700 1 -- 172.21.15.17:0/3521129035 wait complete. 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.134+0000 7f97a2135700 1 Processor -- start 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.134+0000 7f97a2135700 1 -- start start 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.134+0000 7f97a2135700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f979c1021c0 0x7f979c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.134+0000 7f97a2135700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f979c1033c0 0x7f979c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.134+0000 7f97a2135700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f979c103d60 0x7f979c19f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.134+0000 7f97a2135700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c10de10 con 0x7f979c1021c0 2024-03-24T02:56:27.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.134+0000 7f97a2135700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c10dc90 con 0x7f979c1033c0 2024-03-24T02:56:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.134+0000 7f97a2135700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c10df90 con 0x7f979c103d60 2024-03-24T02:56:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f979c1021c0 0x7f979c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:56:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f979c1021c0 0x7f979c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51114/0 (socket says 172.21.15.17:51114) 2024-03-24T02:56:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979b7fe700 1 -- 172.21.15.17:0/1915074971 learned_addr learned my addr 172.21.15.17:0/1915074971 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:56:27.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979affd700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f979c1033c0 0x7f979c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:56:27.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979bfff700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f979c103d60 0x7f979c19f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:56:27.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979b7fe700 1 -- 172.21.15.17:0/1915074971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f979c103d60 msgr2=0x7f979c19f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:27.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979b7fe700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f979c103d60 0x7f979c19f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:27.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979b7fe700 1 -- 172.21.15.17:0/1915074971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f979c1033c0 msgr2=0x7f979c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:27.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979b7fe700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f979c1033c0 0x7f979c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:27.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979b7fe700 1 -- 172.21.15.17:0/1915074971 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c1a0050 con 0x7f979c1021c0 2024-03-24T02:56:27.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979bfff700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f979c103d60 0x7f979c19f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:56:27.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979affd700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f979c1033c0 0x7f979c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:56:27.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f979b7fe700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f979c1021c0 0x7f979c1983f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f978c0057c0 tx=0x7f978c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:56:27.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f9798ff9700 1 -- 172.21.15.17:0/1915074971 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c027070 con 0x7f979c1021c0 2024-03-24T02:56:27.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f97a2135700 1 -- 172.21.15.17:0/1915074971 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c1a02e0 con 0x7f979c1021c0 2024-03-24T02:56:27.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f9798ff9700 1 -- 172.21.15.17:0/1915074971 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f978c0058e0 con 0x7f979c1021c0 2024-03-24T02:56:27.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f9798ff9700 1 -- 172.21.15.17:0/1915074971 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c00f460 con 0x7f979c1021c0 2024-03-24T02:56:27.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.135+0000 7f97a2135700 1 -- 172.21.15.17:0/1915074971 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c1a0890 con 0x7f979c1021c0 2024-03-24T02:56:27.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.136+0000 7f97a2135700 1 -- 172.21.15.17:0/1915074971 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c068e50 con 0x7f979c1021c0 2024-03-24T02:56:27.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.137+0000 7f9798ff9700 1 -- 172.21.15.17:0/1915074971 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f978c02a070 con 0x7f979c1021c0 2024-03-24T02:56:27.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.138+0000 7f9798ff9700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f97840a2910 0x7f97840a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:56:27.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.138+0000 7f979affd700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f97840a2910 0x7f97840a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:56:27.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.138+0000 7f9798ff9700 1 -- 172.21.15.17:0/1915074971 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f978c0cf9e0 con 0x7f979c1021c0 2024-03-24T02:56:27.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.138+0000 7f979affd700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f97840a2910 0x7f97840a4dd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9790005fd0 tx=0x7f9790005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:56:27.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.142+0000 7f9798ff9700 1 -- 172.21.15.17:0/1915074971 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f978c013070 con 0x7f979c1021c0 2024-03-24T02:56:27.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:27.358+0000 7f97a2135700 1 -- 172.21.15.17:0/1915074971 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f979c0625d0 con 0x7f97840a2910 2024-03-24T02:56:27.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:27 smithi017 ceph-mon[31926]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:27.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:27 smithi017 ceph-mon[31926]: osdmap e62: 9 total, 9 up, 9 in 2024-03-24T02:56:27.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:27 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:56:27.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:27 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:56:27.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:27 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:28.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:27 smithi188 ceph-mon[36206]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:28.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:27 smithi188 ceph-mon[36206]: osdmap e62: 9 total, 9 up, 9 in 2024-03-24T02:56:28.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:27 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:56:28.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:27 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:56:28.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:27 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:28.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:27 smithi019 ceph-mon[35944]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:28.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:27 smithi019 ceph-mon[35944]: osdmap e62: 9 total, 9 up, 9 in 2024-03-24T02:56:28.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:27 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:56:28.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:27 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:56:28.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:27 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:28.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:28 smithi017 ceph-mon[31926]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:56:28.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:28 smithi017 ceph-mon[31926]: osdmap e63: 9 total, 9 up, 9 in 2024-03-24T02:56:29.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:28 smithi188 ceph-mon[36206]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:56:29.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:28 smithi188 ceph-mon[36206]: osdmap e63: 9 total, 9 up, 9 in 2024-03-24T02:56:29.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:28 smithi019 ceph-mon[35944]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:56:29.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:28 smithi019 ceph-mon[35944]: osdmap e63: 9 total, 9 up, 9 in 2024-03-24T02:56:29.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:29 smithi017 ceph-mon[31926]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:30.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:29 smithi188 ceph-mon[36206]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:30.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:29 smithi019 ceph-mon[35944]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:30.856 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:30 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2168421486' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a0ca53-b96d-47ed-84f6-b7fc6a75c23f"}]: dispatch 2024-03-24T02:56:30.856 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:30 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2168421486' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50a0ca53-b96d-47ed-84f6-b7fc6a75c23f"}]': finished 2024-03-24T02:56:30.856 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:30 smithi017 ceph-mon[31926]: osdmap e64: 10 total, 9 up, 10 in 2024-03-24T02:56:30.857 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:30 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2168421486' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a0ca53-b96d-47ed-84f6-b7fc6a75c23f"}]: dispatch 2024-03-24T02:56:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:30 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2168421486' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50a0ca53-b96d-47ed-84f6-b7fc6a75c23f"}]': finished 2024-03-24T02:56:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:30 smithi188 ceph-mon[36206]: osdmap e64: 10 total, 9 up, 10 in 2024-03-24T02:56:31.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:30 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2168421486' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a0ca53-b96d-47ed-84f6-b7fc6a75c23f"}]: dispatch 2024-03-24T02:56:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:30 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2168421486' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50a0ca53-b96d-47ed-84f6-b7fc6a75c23f"}]': finished 2024-03-24T02:56:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:30 smithi019 ceph-mon[35944]: osdmap e64: 10 total, 9 up, 10 in 2024-03-24T02:56:31.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:31.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:31 smithi017 ceph-mon[31926]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-03-24T02:56:31.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:31 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1409841494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:56:32.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:31 smithi188 ceph-mon[36206]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-03-24T02:56:32.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:31 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1409841494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:56:32.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:31 smithi019 ceph-mon[35944]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-03-24T02:56:32.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:31 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1409841494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:56:33.942 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:33 smithi017 ceph-mon[31926]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-24T02:56:34.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:33 smithi188 ceph-mon[36206]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-24T02:56:34.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:33 smithi019 ceph-mon[35944]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-24T02:56:35.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:35 smithi017 ceph-mon[31926]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-24T02:56:36.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:35 smithi188 ceph-mon[36206]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-24T02:56:36.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:35 smithi019 ceph-mon[35944]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-24T02:56:37.931 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:37 smithi017 ceph-mon[31926]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T02:56:38.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:37 smithi188 ceph-mon[36206]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T02:56:38.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:37 smithi019 ceph-mon[35944]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T02:56:38.850 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-24T02:56:38.850 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:39.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-24T02:56:39.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:39.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-24T02:56:39.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:39.942 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:39 smithi017 ceph-mon[31926]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-24T02:56:39.943 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:39 smithi017 ceph-mon[31926]: Deploying daemon osd.9 on smithi017 2024-03-24T02:56:40.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:39 smithi188 ceph-mon[36206]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-24T02:56:40.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:39 smithi188 ceph-mon[36206]: Deploying daemon osd.9 on smithi017 2024-03-24T02:56:40.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:39 smithi019 ceph-mon[35944]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-24T02:56:40.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:39 smithi019 ceph-mon[35944]: Deploying daemon osd.9 on smithi017 2024-03-24T02:56:41.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:41 smithi017 ceph-mon[31926]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:42.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:41 smithi188 ceph-mon[36206]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:42.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:41 smithi019 ceph-mon[35944]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:43.780 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:43 smithi017 ceph-mon[31926]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:44.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:43 smithi188 ceph-mon[36206]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:44.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:43 smithi019 ceph-mon[35944]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:45.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:45 smithi019 ceph-mon[35944]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:45.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:45.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:45.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:45.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:45 smithi017 ceph-mon[31926]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:45.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:45.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:45.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:45.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:45 smithi188 ceph-mon[36206]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:45.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:45.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:45.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:47.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:47 smithi017 ceph-mon[31926]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:47.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:47 smithi019 ceph-mon[35944]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:47.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:47 smithi188 ceph-mon[36206]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:47.622 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 9 on host 'smithi017' 2024-03-24T02:56:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.621+0000 7f9798ff9700 1 -- 172.21.15.17:0/1915074971 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f979c0625d0 con 0x7f97840a2910 2024-03-24T02:56:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.623+0000 7f97a2135700 1 -- 172.21.15.17:0/1915074971 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f97840a2910 msgr2=0x7f97840a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.623+0000 7f97a2135700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f97840a2910 0x7f97840a4dd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9790005fd0 tx=0x7f9790005e20 comp rx=0 tx=0).stop 2024-03-24T02:56:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.623+0000 7f97a2135700 1 -- 172.21.15.17:0/1915074971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f979c1021c0 msgr2=0x7f979c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.623+0000 7f97a2135700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f979c1021c0 0x7f979c1983f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f978c0057c0 tx=0x7f978c0057f0 comp rx=0 tx=0).stop 2024-03-24T02:56:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.623+0000 7f97a2135700 1 -- 172.21.15.17:0/1915074971 shutdown_connections 2024-03-24T02:56:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.623+0000 7f97a2135700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f97840a2910 0x7f97840a4dd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.623+0000 7f97a2135700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f979c103d60 0x7f979c19f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.623+0000 7f97a2135700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f979c1021c0 0x7f979c1983f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.623+0000 7f97a2135700 1 --2- 172.21.15.17:0/1915074971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f979c1033c0 0x7f979c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.623+0000 7f97a2135700 1 -- 172.21.15.17:0/1915074971 >> 172.21.15.17:0/1915074971 conn(0x7f979c0fd7a0 msgr2=0x7f979c1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:56:47.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.624+0000 7f97a2135700 1 -- 172.21.15.17:0/1915074971 shutdown_connections 2024-03-24T02:56:47.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:56:47.624+0000 7f97a2135700 1 -- 172.21.15.17:0/1915074971 wait complete. 2024-03-24T02:56:48.274 DEBUG:teuthology.orchestra.run.smithi017:osd.9> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.9.service 2024-03-24T02:56:48.276 INFO:tasks.cephadm:Deploying osd.10 on smithi019 with /dev/vg_nvme/lv_1... 2024-03-24T02:56:48.276 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-24T02:56:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T02:56:48.892 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:48 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:48.892 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:48 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:48.892 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:48 smithi019 ceph-mon[35944]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-24T02:56:48.892 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:48 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:48.892 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:48 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:48.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:48 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:48.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:48 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:48.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:48 smithi017 ceph-mon[31926]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-24T02:56:48.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:48 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:48.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:48 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:49.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:48 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:49.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:48 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:49.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:48 smithi188 ceph-mon[36206]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-24T02:56:49.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:48 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:49.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:48 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:49.880 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:49 smithi019 ceph-mon[35944]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:49.880 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:49 smithi019 ceph-mon[35944]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-24T02:56:49.880 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:49 smithi019 ceph-mon[35944]: osdmap e65: 10 total, 9 up, 10 in 2024-03-24T02:56:49.880 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:49 smithi019 ceph-mon[35944]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-03-24T02:56:49.881 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:49 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:49.922 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:49 smithi017 ceph-mon[31926]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:49.922 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:49 smithi017 ceph-mon[31926]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-24T02:56:49.923 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:49 smithi017 ceph-mon[31926]: osdmap e65: 10 total, 9 up, 10 in 2024-03-24T02:56:49.923 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:49 smithi017 ceph-mon[31926]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-03-24T02:56:49.923 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:49 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:50.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:49 smithi188 ceph-mon[36206]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:50.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:49 smithi188 ceph-mon[36206]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-24T02:56:50.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:49 smithi188 ceph-mon[36206]: osdmap e65: 10 total, 9 up, 10 in 2024-03-24T02:56:50.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:49 smithi188 ceph-mon[36206]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-03-24T02:56:50.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:49 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:50.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 02:56:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T02:56:50.200+0000 7f78d8d4c700 -1 osd.9 0 waiting for initial osdmap 2024-03-24T02:56:50.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 02:56:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T02:56:50.213+0000 7f78d0b37700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T02:56:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-24T02:56:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-24T02:56:50.761 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-03-24T02:56:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-03-24T02:56:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115488 s, 908 MB/s 2024-03-24T02:56:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-03-24T02:56:50.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:50 smithi017 ceph-mon[31926]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-03-24T02:56:50.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:50 smithi017 ceph-mon[31926]: osdmap e66: 10 total, 9 up, 10 in 2024-03-24T02:56:50.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:50 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:50.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:50 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:50.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:50 smithi017 ceph-mon[31926]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' 2024-03-24T02:56:50.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:50 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:51.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:50 smithi188 ceph-mon[36206]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-03-24T02:56:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:50 smithi188 ceph-mon[36206]: osdmap e66: 10 total, 9 up, 10 in 2024-03-24T02:56:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:50 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:50 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:50 smithi188 ceph-mon[36206]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' 2024-03-24T02:56:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:50 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:51.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:50 smithi019 ceph-mon[35944]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-03-24T02:56:51.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:50 smithi019 ceph-mon[35944]: osdmap e66: 10 total, 9 up, 10 in 2024-03-24T02:56:51.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:50 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:51.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:50 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:51.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:50 smithi019 ceph-mon[35944]: from='osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070]' entity='osd.9' 2024-03-24T02:56:51.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:50 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:51.600 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi019:vg_nvme/lv_1 2024-03-24T02:56:51.853 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T02:56:51.955 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:51 smithi017 ceph-mon[31926]: purged_snaps scrub starts 2024-03-24T02:56:51.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:51 smithi017 ceph-mon[31926]: purged_snaps scrub ok 2024-03-24T02:56:51.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:51 smithi017 ceph-mon[31926]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:51.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:51 smithi017 ceph-mon[31926]: osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070] boot 2024-03-24T02:56:51.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:51 smithi017 ceph-mon[31926]: osdmap e67: 10 total, 10 up, 10 in 2024-03-24T02:56:51.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:51 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:51.975 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:51 smithi019 ceph-mon[35944]: purged_snaps scrub starts 2024-03-24T02:56:51.975 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:51 smithi019 ceph-mon[35944]: purged_snaps scrub ok 2024-03-24T02:56:51.975 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:51 smithi019 ceph-mon[35944]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:51.975 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:51 smithi019 ceph-mon[35944]: osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070] boot 2024-03-24T02:56:51.976 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:51 smithi019 ceph-mon[35944]: osdmap e67: 10 total, 10 up, 10 in 2024-03-24T02:56:51.976 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:51 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:52.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:51 smithi188 ceph-mon[36206]: purged_snaps scrub starts 2024-03-24T02:56:52.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:51 smithi188 ceph-mon[36206]: purged_snaps scrub ok 2024-03-24T02:56:52.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:51 smithi188 ceph-mon[36206]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-24T02:56:52.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:51 smithi188 ceph-mon[36206]: osd.9 [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070] boot 2024-03-24T02:56:52.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:51 smithi188 ceph-mon[36206]: osdmap e67: 10 total, 10 up, 10 in 2024-03-24T02:56:52.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:51 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T02:56:53.312 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:56:53.312 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: Detected new or changed devices on smithi017 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi017 to 4273M 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.313 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:53 smithi019 ceph-mon[35944]: osdmap e68: 10 total, 10 up, 10 in 2024-03-24T02:56:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:56:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: Detected new or changed devices on smithi017 2024-03-24T02:56:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi017 to 4273M 2024-03-24T02:56:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:53 smithi017 ceph-mon[31926]: osdmap e68: 10 total, 10 up, 10 in 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: Detected new or changed devices on smithi017 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi017 to 4273M 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:56:53.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:53.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:56:53.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:56:53.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:53 smithi188 ceph-mon[36206]: osdmap e68: 10 total, 10 up, 10 in 2024-03-24T02:56:53.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.652+0000 7fdb21ef8700 1 -- 172.21.15.19:0/19431525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c107eb0 msgr2=0x7fdb1c10a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:53.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.652+0000 7fdb21ef8700 1 --2- 172.21.15.19:0/19431525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c107eb0 0x7fdb1c10a320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009990 tx=0x7fdb14009ca0 comp rx=0 tx=0).stop 2024-03-24T02:56:53.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.652+0000 7fdb21ef8700 1 -- 172.21.15.19:0/19431525 shutdown_connections 2024-03-24T02:56:53.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.652+0000 7fdb21ef8700 1 --2- 172.21.15.19:0/19431525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c107eb0 0x7fdb1c10a320 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:53.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.652+0000 7fdb21ef8700 1 --2- 172.21.15.19:0/19431525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb1c102c50 0x7fdb1c105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.652+0000 7fdb21ef8700 1 --2- 172.21.15.19:0/19431525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb1c105580 0x7fdb1c107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.652+0000 7fdb21ef8700 1 -- 172.21.15.19:0/19431525 >> 172.21.15.19:0/19431525 conn(0x7fdb1c0fc660 msgr2=0x7fdb1c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.652+0000 7fdb21ef8700 1 -- 172.21.15.19:0/19431525 shutdown_connections 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.652+0000 7fdb21ef8700 1 -- 172.21.15.19:0/19431525 wait complete. 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.653+0000 7fdb21ef8700 1 Processor -- start 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.653+0000 7fdb21ef8700 1 -- start start 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.653+0000 7fdb21ef8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb1c102c50 0x7fdb1c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.653+0000 7fdb21ef8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c105580 0x7fdb1c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.653+0000 7fdb21ef8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb1c107eb0 0x7fdb1c19f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.653+0000 7fdb21ef8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c10ddd0 con 0x7fdb1c102c50 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.653+0000 7fdb21ef8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c10dc50 con 0x7fdb1c105580 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.653+0000 7fdb21ef8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c10df50 con 0x7fdb1c107eb0 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.653+0000 7fdb1affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c105580 0x7fdb1c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.653+0000 7fdb1affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c105580 0x7fdb1c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37426/0 (socket says 172.21.15.19:37426) 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.653+0000 7fdb1affd700 1 -- 172.21.15.19:0/4145371950 learned_addr learned my addr 172.21.15.19:0/4145371950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-24T02:56:53.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb1b7fe700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb1c102c50 0x7fdb1c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:56:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb1bfff700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb1c107eb0 0x7fdb1c19f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:56:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb1affd700 1 -- 172.21.15.19:0/4145371950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb1c107eb0 msgr2=0x7fdb1c19f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb1affd700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb1c107eb0 0x7fdb1c19f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb1affd700 1 -- 172.21.15.19:0/4145371950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb1c102c50 msgr2=0x7fdb1c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:56:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb1affd700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb1c102c50 0x7fdb1c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:56:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb1affd700 1 -- 172.21.15.19:0/4145371950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1c1a0040 con 0x7fdb1c105580 2024-03-24T02:56:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb1affd700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c105580 0x7fdb1c198920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400d950 tx=0x7fdb0400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:56:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb18ff9700 1 -- 172.21.15.19:0/4145371950 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04016070 con 0x7fdb1c105580 2024-03-24T02:56:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb18ff9700 1 -- 172.21.15.19:0/4145371950 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb0400fd70 con 0x7fdb1c105580 2024-03-24T02:56:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb21ef8700 1 -- 172.21.15.19:0/4145371950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1c1a0330 con 0x7fdb1c105580 2024-03-24T02:56:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb18ff9700 1 -- 172.21.15.19:0/4145371950 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0400e610 con 0x7fdb1c105580 2024-03-24T02:56:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.654+0000 7fdb21ef8700 1 -- 172.21.15.19:0/4145371950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1c1a0870 con 0x7fdb1c105580 2024-03-24T02:56:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.655+0000 7fdb21ef8700 1 -- 172.21.15.19:0/4145371950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1c068e50 con 0x7fdb1c105580 2024-03-24T02:56:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.656+0000 7fdb18ff9700 1 -- 172.21.15.19:0/4145371950 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdb04019070 con 0x7fdb1c105580 2024-03-24T02:56:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.657+0000 7fdb18ff9700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdb080a2890 0x7fdb080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:56:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.657+0000 7fdb18ff9700 1 -- 172.21.15.19:0/4145371950 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fdb040cdd90 con 0x7fdb1c105580 2024-03-24T02:56:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.657+0000 7fdb1b7fe700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdb080a2890 0x7fdb080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:56:53.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.658+0000 7fdb1b7fe700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdb080a2890 0x7fdb080a4d50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0060b0 tx=0x7fdb0c00c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:56:53.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.661+0000 7fdb18ff9700 1 -- 172.21.15.19:0/4145371950 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb0409b590 con 0x7fdb1c105580 2024-03-24T02:56:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:56:53.877+0000 7fdb21ef8700 1 -- 172.21.15.19:0/4145371950 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb1c062690 con 0x7fdb080a2890 2024-03-24T02:56:54.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:54 smithi017 ceph-mon[31926]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:56:54.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:54 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:56:54.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:54 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:56:54.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:54 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:54.509 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:54 smithi019 ceph-mon[35944]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:56:54.509 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:54 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:56:54.509 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:54 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:56:54.509 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:54 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:54.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:54 smithi188 ceph-mon[36206]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:56:54.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:54 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:56:54.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:54 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:56:54.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:54 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:56:55.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:55 smithi017 ceph-mon[31926]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:56:55.508 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:55 smithi019 ceph-mon[35944]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:56:55.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:55 smithi188 ceph-mon[36206]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:56:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:57 smithi019 ceph-mon[35944]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:56:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:57 smithi019 ceph-mon[35944]: from='client.? 172.21.15.19:0/718612144' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e702b87a-9535-42db-aa8f-58f35cecb5d9"}]: dispatch 2024-03-24T02:56:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:57 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e702b87a-9535-42db-aa8f-58f35cecb5d9"}]: dispatch 2024-03-24T02:56:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:57 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e702b87a-9535-42db-aa8f-58f35cecb5d9"}]': finished 2024-03-24T02:56:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:57 smithi019 ceph-mon[35944]: osdmap e69: 11 total, 10 up, 11 in 2024-03-24T02:56:57.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:57 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:56:57.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:57 smithi017 ceph-mon[31926]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:56:57.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:57 smithi017 ceph-mon[31926]: from='client.? 172.21.15.19:0/718612144' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e702b87a-9535-42db-aa8f-58f35cecb5d9"}]: dispatch 2024-03-24T02:56:57.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:57 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e702b87a-9535-42db-aa8f-58f35cecb5d9"}]: dispatch 2024-03-24T02:56:57.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:57 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e702b87a-9535-42db-aa8f-58f35cecb5d9"}]': finished 2024-03-24T02:56:57.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:57 smithi017 ceph-mon[31926]: osdmap e69: 11 total, 10 up, 11 in 2024-03-24T02:56:57.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:57 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:56:57.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:57 smithi188 ceph-mon[36206]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:56:57.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:57 smithi188 ceph-mon[36206]: from='client.? 172.21.15.19:0/718612144' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e702b87a-9535-42db-aa8f-58f35cecb5d9"}]: dispatch 2024-03-24T02:56:57.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:57 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e702b87a-9535-42db-aa8f-58f35cecb5d9"}]: dispatch 2024-03-24T02:56:57.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:57 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e702b87a-9535-42db-aa8f-58f35cecb5d9"}]': finished 2024-03-24T02:56:57.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:57 smithi188 ceph-mon[36206]: osdmap e69: 11 total, 10 up, 11 in 2024-03-24T02:56:57.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:57 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:56:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:58 smithi017 ceph-mon[31926]: from='client.? 172.21.15.19:0/2513555167' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:56:58.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:58 smithi188 ceph-mon[36206]: from='client.? 172.21.15.19:0/2513555167' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:56:58.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:58 smithi019 ceph-mon[35944]: from='client.? 172.21.15.19:0/2513555167' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:56:59.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:56:59 smithi017 ceph-mon[31926]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:56:59.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:56:59 smithi188 ceph-mon[36206]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:56:59.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:56:59 smithi019 ceph-mon[35944]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:01.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:01 smithi019 ceph-mon[35944]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:01.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:01 smithi017 ceph-mon[31926]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:01.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:01 smithi188 ceph-mon[36206]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:03.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:03 smithi019 ceph-mon[35944]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:03.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:03 smithi017 ceph-mon[31926]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:03.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:03 smithi188 ceph-mon[36206]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:05 smithi017 ceph-mon[31926]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-24T02:57:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:05 smithi017 ceph-mon[31926]: Deploying daemon osd.10 on smithi019 2024-03-24T02:57:05.489 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:05 smithi019 ceph-mon[35944]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:05.490 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-24T02:57:05.490 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:05.490 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:05 smithi019 ceph-mon[35944]: Deploying daemon osd.10 on smithi019 2024-03-24T02:57:05.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:05 smithi188 ceph-mon[36206]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:05.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-24T02:57:05.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:05.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:05 smithi188 ceph-mon[36206]: Deploying daemon osd.10 on smithi019 2024-03-24T02:57:07.296 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:07 smithi019 ceph-mon[35944]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:07.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:07 smithi017 ceph-mon[31926]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:07 smithi188 ceph-mon[36206]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:09.265 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:09 smithi019 ceph-mon[35944]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:09.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:09 smithi017 ceph-mon[31926]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:09.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:09 smithi188 ceph-mon[36206]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:11.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:10 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:11.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:10 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:11.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:10 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:10 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:10 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:11.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:10 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:11.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:10 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:11.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:10 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:11.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:10 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:12.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:11 smithi019 ceph-mon[35944]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:12.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:11 smithi017 ceph-mon[31926]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:12.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:11 smithi188 ceph-mon[36206]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:13.794 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 10 on host 'smithi019' 2024-03-24T02:57:13.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.794+0000 7fdb18ff9700 1 -- 172.21.15.19:0/4145371950 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fdb1c062690 con 0x7fdb080a2890 2024-03-24T02:57:13.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.796+0000 7fdb21ef8700 1 -- 172.21.15.19:0/4145371950 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdb080a2890 msgr2=0x7fdb080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:13.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.796+0000 7fdb21ef8700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdb080a2890 0x7fdb080a4d50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0060b0 tx=0x7fdb0c00c040 comp rx=0 tx=0).stop 2024-03-24T02:57:13.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.796+0000 7fdb21ef8700 1 -- 172.21.15.19:0/4145371950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c105580 msgr2=0x7fdb1c198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:13.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.796+0000 7fdb21ef8700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c105580 0x7fdb1c198920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400d950 tx=0x7fdb0400dc60 comp rx=0 tx=0).stop 2024-03-24T02:57:13.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.797+0000 7fdb21ef8700 1 -- 172.21.15.19:0/4145371950 shutdown_connections 2024-03-24T02:57:13.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.797+0000 7fdb21ef8700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdb080a2890 0x7fdb080a4d50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:13.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.797+0000 7fdb21ef8700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb1c107eb0 0x7fdb1c19f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:13.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.797+0000 7fdb21ef8700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb1c102c50 0x7fdb1c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:13.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.797+0000 7fdb21ef8700 1 --2- 172.21.15.19:0/4145371950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c105580 0x7fdb1c198920 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:13.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.797+0000 7fdb21ef8700 1 -- 172.21.15.19:0/4145371950 >> 172.21.15.19:0/4145371950 conn(0x7fdb1c0fc660 msgr2=0x7fdb1c1035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:13.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.797+0000 7fdb21ef8700 1 -- 172.21.15.19:0/4145371950 shutdown_connections 2024-03-24T02:57:13.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-24T02:57:13.797+0000 7fdb21ef8700 1 -- 172.21.15.19:0/4145371950 wait complete. 2024-03-24T02:57:14.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:13 smithi019 ceph-mon[35944]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:14.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:13 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:13 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:13 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:14.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:13 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:14.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:13 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:13 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:13 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:13 smithi019 ceph-mon[35944]: from='osd.10 [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-24T02:57:14.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:13 smithi019 ceph-mon[35944]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-24T02:57:14.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:13 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:13 smithi017 ceph-mon[31926]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:13 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:13 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:13 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:13 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:13 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:13 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:13 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:13 smithi017 ceph-mon[31926]: from='osd.10 [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-24T02:57:14.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:13 smithi017 ceph-mon[31926]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-24T02:57:14.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:13 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:14.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:13 smithi188 ceph-mon[36206]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:14.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:13 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:13 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:13 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:14.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:13 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:14.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:13 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:13 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:13 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:14.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:13 smithi188 ceph-mon[36206]: from='osd.10 [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-24T02:57:14.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:13 smithi188 ceph-mon[36206]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-24T02:57:14.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:13 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:14.472 DEBUG:teuthology.orchestra.run.smithi019:osd.10> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.10.service 2024-03-24T02:57:14.475 INFO:tasks.cephadm:Deploying osd.11 on smithi188 with /dev/vg_nvme/lv_1... 2024-03-24T02:57:14.475 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-24T02:57:14.726 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T02:57:15.780 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:15 smithi019 ceph-mon[35944]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:15.780 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:15 smithi019 ceph-mon[35944]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-24T02:57:15.780 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:15 smithi019 ceph-mon[35944]: from='osd.10 [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:57:15.780 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:15 smithi019 ceph-mon[35944]: osdmap e70: 11 total, 10 up, 11 in 2024-03-24T02:57:15.780 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:15 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:15.780 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:15 smithi019 ceph-mon[35944]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:57:15.780 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 02:57:15 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T02:57:15.762+0000 7f7bedda3700 -1 osd.10 0 waiting for initial osdmap 2024-03-24T02:57:15.780 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 02:57:15 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T02:57:15.779+0000 7f7be9b96700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T02:57:15.986 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:15 smithi188 ceph-mon[36206]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:15.986 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:15 smithi188 ceph-mon[36206]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-24T02:57:15.986 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:15 smithi188 ceph-mon[36206]: from='osd.10 [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:57:15.986 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:15 smithi188 ceph-mon[36206]: osdmap e70: 11 total, 10 up, 11 in 2024-03-24T02:57:15.986 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:15 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:15.987 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:15 smithi188 ceph-mon[36206]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:57:16.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:15 smithi017 ceph-mon[31926]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:16.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:15 smithi017 ceph-mon[31926]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-24T02:57:16.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:15 smithi017 ceph-mon[31926]: from='osd.10 [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:57:16.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:15 smithi017 ceph-mon[31926]: osdmap e70: 11 total, 10 up, 11 in 2024-03-24T02:57:16.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:15 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:16.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:15 smithi017 ceph-mon[31926]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-24T02:57:16.873 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:16 smithi188 ceph-mon[36206]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-24T02:57:16.873 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:16 smithi188 ceph-mon[36206]: osdmap e71: 11 total, 10 up, 11 in 2024-03-24T02:57:16.873 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:16 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:16.873 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:16 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:16.873 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-24T02:57:16.874 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-24T02:57:16.874 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-03-24T02:57:16.874 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-03-24T02:57:16.874 INFO:teuthology.orchestra.run.smithi188.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011365 s, 923 MB/s 2024-03-24T02:57:16.874 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-03-24T02:57:17.040 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:16 smithi019 ceph-mon[35944]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-24T02:57:17.040 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:16 smithi019 ceph-mon[35944]: osdmap e71: 11 total, 10 up, 11 in 2024-03-24T02:57:17.040 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:16 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:17.041 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:16 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:17.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:16 smithi017 ceph-mon[31926]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-24T02:57:17.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:16 smithi017 ceph-mon[31926]: osdmap e71: 11 total, 10 up, 11 in 2024-03-24T02:57:17.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:16 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:17.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:16 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:17.659 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi188:vg_nvme/lv_1 2024-03-24T02:57:17.899 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T02:57:18.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: purged_snaps scrub starts 2024-03-24T02:57:18.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: purged_snaps scrub ok 2024-03-24T02:57:18.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:18.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:18.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: osd.10 [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892] boot 2024-03-24T02:57:18.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: osdmap e72: 11 total, 11 up, 11 in 2024-03-24T02:57:18.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:18.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:18.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:18.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:18.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:18.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:18.022 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:18.022 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:18.022 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:18.022 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:18.022 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:18.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: purged_snaps scrub starts 2024-03-24T02:57:18.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: purged_snaps scrub ok 2024-03-24T02:57:18.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:18.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:18.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: osd.10 [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892] boot 2024-03-24T02:57:18.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: osdmap e72: 11 total, 11 up, 11 in 2024-03-24T02:57:18.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:18.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:18.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:18.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:18.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:18.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:18.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:18.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:18.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:18.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:18.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: purged_snaps scrub starts 2024-03-24T02:57:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: purged_snaps scrub ok 2024-03-24T02:57:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-24T02:57:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: osd.10 [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892] boot 2024-03-24T02:57:18.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: osdmap e72: 11 total, 11 up, 11 in 2024-03-24T02:57:18.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T02:57:18.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:18.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:18.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:18.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:18.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:18.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:18.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:18.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:18.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:18.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:18.978 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:18 smithi188 ceph-mon[36206]: Detected new or changed devices on smithi019 2024-03-24T02:57:18.979 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:18 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi019 to 4271M 2024-03-24T02:57:18.979 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:18 smithi188 ceph-mon[36206]: osdmap e73: 11 total, 11 up, 11 in 2024-03-24T02:57:19.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:18 smithi019 ceph-mon[35944]: Detected new or changed devices on smithi019 2024-03-24T02:57:19.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:18 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi019 to 4271M 2024-03-24T02:57:19.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:18 smithi019 ceph-mon[35944]: osdmap e73: 11 total, 11 up, 11 in 2024-03-24T02:57:19.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:18 smithi017 ceph-mon[31926]: Detected new or changed devices on smithi019 2024-03-24T02:57:19.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:18 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi019 to 4271M 2024-03-24T02:57:19.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:18 smithi017 ceph-mon[31926]: osdmap e73: 11 total, 11 up, 11 in 2024-03-24T02:57:19.520 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.519+0000 7f62f97e1700 1 -- 172.21.15.188:0/4113911910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f4103e50 msgr2=0x7f62f410a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.519+0000 7f62f97e1700 1 --2- 172.21.15.188:0/4113911910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f4103e50 0x7f62f410a700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00b0a0 tx=0x7f62ec00b3b0 comp rx=0 tx=0).stop 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.519+0000 7f62f97e1700 1 -- 172.21.15.188:0/4113911910 shutdown_connections 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.519+0000 7f62f97e1700 1 --2- 172.21.15.188:0/4113911910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f4103e50 0x7f62f410a700 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.519+0000 7f62f97e1700 1 --2- 172.21.15.188:0/4113911910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62f41004d0 0x7f62f41008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.519+0000 7f62f97e1700 1 --2- 172.21.15.188:0/4113911910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f41034b0 0x7f62f4103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.519+0000 7f62f97e1700 1 -- 172.21.15.188:0/4113911910 >> 172.21.15.188:0/4113911910 conn(0x7f62f40fbe30 msgr2=0x7f62f40fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.520+0000 7f62f97e1700 1 -- 172.21.15.188:0/4113911910 shutdown_connections 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.520+0000 7f62f97e1700 1 -- 172.21.15.188:0/4113911910 wait complete. 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.520+0000 7f62f97e1700 1 Processor -- start 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.520+0000 7f62f97e1700 1 -- start start 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.520+0000 7f62f97e1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f41004d0 0x7f62f4102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f97e1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f41034b0 0x7f62f4101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:19.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f97e1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62f4103e50 0x7f62f41019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f97e1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f410de50 con 0x7f62f4103e50 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f97e1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f410dcd0 con 0x7f62f41034b0 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f97e1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f410dfd0 con 0x7f62f41004d0 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f41004d0 0x7f62f4102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f41004d0 0x7f62f4102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:48658/0 (socket says 172.21.15.188:48658) 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f2ffd700 1 -- 172.21.15.188:0/4087637856 learned_addr learned my addr 172.21.15.188:0/4087637856 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f27fc700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f41034b0 0x7f62f4101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f37fe700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62f4103e50 0x7f62f41019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f2ffd700 1 -- 172.21.15.188:0/4087637856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f41034b0 msgr2=0x7f62f4101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f2ffd700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f41034b0 0x7f62f4101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f2ffd700 1 -- 172.21.15.188:0/4087637856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62f4103e50 msgr2=0x7f62f41019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f2ffd700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62f4103e50 0x7f62f41019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:19.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f2ffd700 1 -- 172.21.15.188:0/4087637856 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f4102250 con 0x7f62f41004d0 2024-03-24T02:57:19.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f27fc700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f41034b0 0x7f62f4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:57:19.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f2ffd700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f41004d0 0x7f62f4102e10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f62e400d980 tx=0x7f62e400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:19.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62dbfff700 1 -- 172.21.15.188:0/4087637856 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4017030 con 0x7f62f41004d0 2024-03-24T02:57:19.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f97e1700 1 -- 172.21.15.188:0/4087637856 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f41a2530 con 0x7f62f41004d0 2024-03-24T02:57:19.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62dbfff700 1 -- 172.21.15.188:0/4087637856 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62e4004500 con 0x7f62f41004d0 2024-03-24T02:57:19.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62dbfff700 1 -- 172.21.15.188:0/4087637856 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4003aa0 con 0x7f62f41004d0 2024-03-24T02:57:19.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.521+0000 7f62f97e1700 1 -- 172.21.15.188:0/4087637856 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f41a2a70 con 0x7f62f41004d0 2024-03-24T02:57:19.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.522+0000 7f62f97e1700 1 -- 172.21.15.188:0/4087637856 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f4068e50 con 0x7f62f41004d0 2024-03-24T02:57:19.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.523+0000 7f62dbfff700 1 -- 172.21.15.188:0/4087637856 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f62e401a070 con 0x7f62f41004d0 2024-03-24T02:57:19.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.524+0000 7f62dbfff700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f62dc0a2890 0x7f62dc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:19.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.524+0000 7f62dbfff700 1 -- 172.21.15.188:0/4087637856 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f62e40cddc0 con 0x7f62f41004d0 2024-03-24T02:57:19.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.524+0000 7f62f27fc700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f62dc0a2890 0x7f62dc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:19.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.525+0000 7f62f27fc700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f62dc0a2890 0x7f62dc0a4d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62e80099d0 tx=0x7f62e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:19.527 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.529+0000 7f62dbfff700 1 -- 172.21.15.188:0/4087637856 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f62e40232f0 con 0x7f62f41004d0 2024-03-24T02:57:19.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:19.734+0000 7f62f97e1700 1 -- 172.21.15.188:0/4087637856 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f62f40625d0 con 0x7f62dc0a2890 2024-03-24T02:57:20.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:19 smithi188 ceph-mon[36206]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:20.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:19 smithi188 ceph-mon[36206]: osdmap e74: 11 total, 11 up, 11 in 2024-03-24T02:57:20.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:19 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:57:20.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:19 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:57:20.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:19 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:20.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:19 smithi019 ceph-mon[35944]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:20.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:19 smithi019 ceph-mon[35944]: osdmap e74: 11 total, 11 up, 11 in 2024-03-24T02:57:20.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:19 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:57:20.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:19 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:57:20.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:19 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:19 smithi017 ceph-mon[31926]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:19 smithi017 ceph-mon[31926]: osdmap e74: 11 total, 11 up, 11 in 2024-03-24T02:57:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:19 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T02:57:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:19 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T02:57:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:19 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:21.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:20 smithi188 ceph-mon[36206]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:57:21.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:20 smithi188 ceph-mon[36206]: osdmap e75: 11 total, 11 up, 11 in 2024-03-24T02:57:21.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:20 smithi019 ceph-mon[35944]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:57:21.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:20 smithi019 ceph-mon[35944]: osdmap e75: 11 total, 11 up, 11 in 2024-03-24T02:57:21.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:20 smithi017 ceph-mon[31926]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:57:21.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:20 smithi017 ceph-mon[31926]: osdmap e75: 11 total, 11 up, 11 in 2024-03-24T02:57:22.006 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:21 smithi188 ceph-mon[36206]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-24T02:57:22.007 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:21 smithi188 ceph-mon[36206]: osdmap e76: 11 total, 11 up, 11 in 2024-03-24T02:57:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:21 smithi019 ceph-mon[35944]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-24T02:57:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:21 smithi019 ceph-mon[35944]: osdmap e76: 11 total, 11 up, 11 in 2024-03-24T02:57:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:21 smithi017 ceph-mon[31926]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-24T02:57:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:21 smithi017 ceph-mon[31926]: osdmap e76: 11 total, 11 up, 11 in 2024-03-24T02:57:24.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:23 smithi188 ceph-mon[36206]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-03-24T02:57:24.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:23 smithi188 ceph-mon[36206]: from='client.? 172.21.15.188:0/3825132864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb1fc489-8371-4d59-9086-2629adec0374"}]: dispatch 2024-03-24T02:57:24.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:23 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb1fc489-8371-4d59-9086-2629adec0374"}]: dispatch 2024-03-24T02:57:24.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:23 smithi188 ceph-mon[36206]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb1fc489-8371-4d59-9086-2629adec0374"}]': finished 2024-03-24T02:57:24.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:23 smithi188 ceph-mon[36206]: osdmap e77: 12 total, 11 up, 12 in 2024-03-24T02:57:24.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:23 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:24.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:23 smithi188 ceph-mon[36206]: from='client.? 172.21.15.188:0/4092122781' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:57:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:23 smithi019 ceph-mon[35944]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-03-24T02:57:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:23 smithi019 ceph-mon[35944]: from='client.? 172.21.15.188:0/3825132864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb1fc489-8371-4d59-9086-2629adec0374"}]: dispatch 2024-03-24T02:57:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:23 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb1fc489-8371-4d59-9086-2629adec0374"}]: dispatch 2024-03-24T02:57:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:23 smithi019 ceph-mon[35944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb1fc489-8371-4d59-9086-2629adec0374"}]': finished 2024-03-24T02:57:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:23 smithi019 ceph-mon[35944]: osdmap e77: 12 total, 11 up, 12 in 2024-03-24T02:57:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:23 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:24.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:23 smithi019 ceph-mon[35944]: from='client.? 172.21.15.188:0/4092122781' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:57:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:23 smithi017 ceph-mon[31926]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-03-24T02:57:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:23 smithi017 ceph-mon[31926]: from='client.? 172.21.15.188:0/3825132864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb1fc489-8371-4d59-9086-2629adec0374"}]: dispatch 2024-03-24T02:57:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:23 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb1fc489-8371-4d59-9086-2629adec0374"}]: dispatch 2024-03-24T02:57:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:23 smithi017 ceph-mon[31926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb1fc489-8371-4d59-9086-2629adec0374"}]': finished 2024-03-24T02:57:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:23 smithi017 ceph-mon[31926]: osdmap e77: 12 total, 11 up, 12 in 2024-03-24T02:57:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:23 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:23 smithi017 ceph-mon[31926]: from='client.? 172.21.15.188:0/4092122781' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T02:57:26.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:25 smithi188 ceph-mon[36206]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 354 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-24T02:57:26.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:25 smithi019 ceph-mon[35944]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 354 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-24T02:57:26.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:25 smithi017 ceph-mon[31926]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 354 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-24T02:57:27.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:27 smithi188 ceph-mon[36206]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:28.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:27 smithi019 ceph-mon[35944]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:28.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:27 smithi017 ceph-mon[31926]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:30.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:29 smithi019 ceph-mon[35944]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:30.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:29 smithi017 ceph-mon[31926]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:30.294 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:29 smithi188 ceph-mon[36206]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:31.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-24T02:57:31.065 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-24T02:57:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:31.107 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-24T02:57:31.107 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:31.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:31 smithi188 ceph-mon[36206]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:31.952 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:31 smithi188 ceph-mon[36206]: Deploying daemon osd.11 on smithi188 2024-03-24T02:57:32.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:31 smithi019 ceph-mon[35944]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:32.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:31 smithi019 ceph-mon[35944]: Deploying daemon osd.11 on smithi188 2024-03-24T02:57:32.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:31 smithi017 ceph-mon[31926]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:32.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:31 smithi017 ceph-mon[31926]: Deploying daemon osd.11 on smithi188 2024-03-24T02:57:34.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:33 smithi019 ceph-mon[35944]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:34.080 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:33 smithi188 ceph-mon[36206]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:33 smithi017 ceph-mon[31926]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:36.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:35 smithi019 ceph-mon[35944]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:36.117 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:35 smithi188 ceph-mon[36206]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:36.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:35 smithi017 ceph-mon[31926]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:37.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:37 smithi017 ceph-mon[31926]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:37.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:37 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:37.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:37 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:37.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:37 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:37.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:37 smithi188 ceph-mon[36206]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:37.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:37 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:37.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:37 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:37.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:37 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:37.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:37 smithi019 ceph-mon[35944]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:37.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:37 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:37.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:37 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:37.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:37 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:39.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:39 smithi017 ceph-mon[31926]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:39.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:39 smithi188 ceph-mon[36206]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:39.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:39 smithi019 ceph-mon[35944]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:40.051 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 11 on host 'smithi188' 2024-03-24T02:57:40.051 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.050+0000 7f62dbfff700 1 -- 172.21.15.188:0/4087637856 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f62f40625d0 con 0x7f62dc0a2890 2024-03-24T02:57:40.053 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.053+0000 7f62f97e1700 1 -- 172.21.15.188:0/4087637856 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f62dc0a2890 msgr2=0x7f62dc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:40.053 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.053+0000 7f62f97e1700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f62dc0a2890 0x7f62dc0a4d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62e80099d0 tx=0x7f62e8008040 comp rx=0 tx=0).stop 2024-03-24T02:57:40.053 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.053+0000 7f62f97e1700 1 -- 172.21.15.188:0/4087637856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f41004d0 msgr2=0x7f62f4102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:40.053 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.053+0000 7f62f97e1700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f41004d0 0x7f62f4102e10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f62e400d980 tx=0x7f62e400dc90 comp rx=0 tx=0).stop 2024-03-24T02:57:40.053 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.053+0000 7f62f97e1700 1 -- 172.21.15.188:0/4087637856 shutdown_connections 2024-03-24T02:57:40.053 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.053+0000 7f62f97e1700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f62dc0a2890 0x7f62dc0a4d50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:40.053 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.053+0000 7f62f97e1700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62f4103e50 0x7f62f41019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:40.053 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.053+0000 7f62f97e1700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f41004d0 0x7f62f4102e10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:40.053 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.053+0000 7f62f97e1700 1 --2- 172.21.15.188:0/4087637856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f41034b0 0x7f62f4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:40.054 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.053+0000 7f62f97e1700 1 -- 172.21.15.188:0/4087637856 >> 172.21.15.188:0/4087637856 conn(0x7f62f40fbe30 msgr2=0x7f62f4106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:40.054 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.053+0000 7f62f97e1700 1 -- 172.21.15.188:0/4087637856 shutdown_connections 2024-03-24T02:57:40.054 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:40.053+0000 7f62f97e1700 1 -- 172.21.15.188:0/4087637856 wait complete. 2024-03-24T02:57:40.718 DEBUG:teuthology.orchestra.run.smithi188:osd.11> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.11.service 2024-03-24T02:57:40.720 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-24T02:57:40.721 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-24T02:57:40.976 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:57:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:41 smithi188 ceph-mon[36206]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:41 smithi188 ceph-mon[36206]: from='osd.11 [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-24T02:57:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:41 smithi188 ceph-mon[36206]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-24T02:57:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:41 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:41 smithi019 ceph-mon[35944]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:41 smithi019 ceph-mon[35944]: from='osd.11 [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-24T02:57:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:41 smithi019 ceph-mon[35944]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-24T02:57:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:41.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:41 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:41.336 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:41 smithi017 ceph-mon[31926]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:41.336 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:41.336 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:41.336 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:41 smithi017 ceph-mon[31926]: from='osd.11 [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-24T02:57:41.336 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:41 smithi017 ceph-mon[31926]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-24T02:57:41.336 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:41.336 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:41 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:42.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:42 smithi188 ceph-mon[36206]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-24T02:57:42.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:42 smithi188 ceph-mon[36206]: osdmap e78: 12 total, 11 up, 12 in 2024-03-24T02:57:42.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:42 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:42.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:42 smithi188 ceph-mon[36206]: from='osd.11 [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:57:42.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:42 smithi188 ceph-mon[36206]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:57:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:42 smithi019 ceph-mon[35944]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-24T02:57:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:42 smithi019 ceph-mon[35944]: osdmap e78: 12 total, 11 up, 12 in 2024-03-24T02:57:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:42 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:42 smithi019 ceph-mon[35944]: from='osd.11 [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:57:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:42 smithi019 ceph-mon[35944]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:57:42.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:42 smithi017 ceph-mon[31926]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-24T02:57:42.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:42 smithi017 ceph-mon[31926]: osdmap e78: 12 total, 11 up, 12 in 2024-03-24T02:57:42.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:42 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:42.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:42 smithi017 ceph-mon[31926]: from='osd.11 [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:57:42.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:42 smithi017 ceph-mon[31926]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-03-24T02:57:42.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.785+0000 7fec5de8e700 1 -- 172.21.15.17:0/4038280849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec581004d0 msgr2=0x7fec581008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.785+0000 7fec5de8e700 1 --2- 172.21.15.17:0/4038280849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec581004d0 0x7fec581008f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fec48009990 tx=0x7fec48009ca0 comp rx=0 tx=0).stop 2024-03-24T02:57:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.785+0000 7fec5de8e700 1 -- 172.21.15.17:0/4038280849 shutdown_connections 2024-03-24T02:57:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.785+0000 7fec5de8e700 1 --2- 172.21.15.17:0/4038280849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec58103ef0 0x7fec5810a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.785+0000 7fec5de8e700 1 --2- 172.21.15.17:0/4038280849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec581004d0 0x7fec581008f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.785+0000 7fec5de8e700 1 --2- 172.21.15.17:0/4038280849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec58103550 0x7fec581039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.785+0000 7fec5de8e700 1 -- 172.21.15.17:0/4038280849 >> 172.21.15.17:0/4038280849 conn(0x7fec580fbe30 msgr2=0x7fec580fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.785+0000 7fec5de8e700 1 -- 172.21.15.17:0/4038280849 shutdown_connections 2024-03-24T02:57:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.785+0000 7fec5de8e700 1 -- 172.21.15.17:0/4038280849 wait complete. 2024-03-24T02:57:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.785+0000 7fec5de8e700 1 Processor -- start 2024-03-24T02:57:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.785+0000 7fec5de8e700 1 -- start start 2024-03-24T02:57:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec5de8e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec581004d0 0x7fec58102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec5de8e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec58103550 0x7fec58101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec5de8e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec58103ef0 0x7fec58101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec5de8e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5810dde0 con 0x7fec58103ef0 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec5de8e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5810dc60 con 0x7fec581004d0 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec5de8e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5810df60 con 0x7fec58103550 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec57fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec58103ef0 0x7fec58101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec57fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec58103ef0 0x7fec58101a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47160/0 (socket says 172.21.15.17:47160) 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec57fff700 1 -- 172.21.15.17:0/2051734786 learned_addr learned my addr 172.21.15.17:0/2051734786 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec56ffd700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec58103550 0x7fec58101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec577fe700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec581004d0 0x7fec58102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec57fff700 1 -- 172.21.15.17:0/2051734786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec58103550 msgr2=0x7fec58101500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec57fff700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec58103550 0x7fec58101500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec57fff700 1 -- 172.21.15.17:0/2051734786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec581004d0 msgr2=0x7fec58102eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec57fff700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec581004d0 0x7fec58102eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec57fff700 1 -- 172.21.15.17:0/2051734786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec581022f0 con 0x7fec58103ef0 2024-03-24T02:57:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec57fff700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec58103ef0 0x7fec58101a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fec50007d70 tx=0x7fec5000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec54ff9700 1 -- 172.21.15.17:0/2051734786 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec50016020 con 0x7fec58103ef0 2024-03-24T02:57:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.786+0000 7fec5de8e700 1 -- 172.21.15.17:0/2051734786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec581a2460 con 0x7fec58103ef0 2024-03-24T02:57:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.787+0000 7fec54ff9700 1 -- 172.21.15.17:0/2051734786 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec5000ac90 con 0x7fec58103ef0 2024-03-24T02:57:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.787+0000 7fec54ff9700 1 -- 172.21.15.17:0/2051734786 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec500115c0 con 0x7fec58103ef0 2024-03-24T02:57:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.787+0000 7fec5de8e700 1 -- 172.21.15.17:0/2051734786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec581a2970 con 0x7fec58103ef0 2024-03-24T02:57:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.787+0000 7fec5de8e700 1 -- 172.21.15.17:0/2051734786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec58068e50 con 0x7fec58103ef0 2024-03-24T02:57:42.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.789+0000 7fec54ff9700 1 -- 172.21.15.17:0/2051734786 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fec50019020 con 0x7fec58103ef0 2024-03-24T02:57:42.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.790+0000 7fec54ff9700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fec400a2910 0x7fec400a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:42.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.790+0000 7fec577fe700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fec400a2910 0x7fec400a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:42.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.790+0000 7fec54ff9700 1 -- 172.21.15.17:0/2051734786 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fec500ce7e0 con 0x7fec58103ef0 2024-03-24T02:57:42.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.790+0000 7fec577fe700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fec400a2910 0x7fec400a4dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fec48003890 tx=0x7fec48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:42.793+0000 7fec54ff9700 1 -- 172.21.15.17:0/2051734786 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fec50020020 con 0x7fec58103ef0 2024-03-24T02:57:43.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.003+0000 7fec5de8e700 1 -- 172.21.15.17:0/2051734786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fec5802da30 con 0x7fec58103ef0 2024-03-24T02:57:43.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.003+0000 7fec54ff9700 1 -- 172.21.15.17:0/2051734786 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7fec5009be40 con 0x7fec58103ef0 2024-03-24T02:57:43.004 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:57:43.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.005+0000 7fec5de8e700 1 -- 172.21.15.17:0/2051734786 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fec400a2910 msgr2=0x7fec400a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:43.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.005+0000 7fec5de8e700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fec400a2910 0x7fec400a4dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fec48003890 tx=0x7fec48005fd0 comp rx=0 tx=0).stop 2024-03-24T02:57:43.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.005+0000 7fec5de8e700 1 -- 172.21.15.17:0/2051734786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec58103ef0 msgr2=0x7fec58101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:43.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.005+0000 7fec5de8e700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec58103ef0 0x7fec58101a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fec50007d70 tx=0x7fec5000a340 comp rx=0 tx=0).stop 2024-03-24T02:57:43.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.005+0000 7fec5de8e700 1 -- 172.21.15.17:0/2051734786 shutdown_connections 2024-03-24T02:57:43.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.005+0000 7fec5de8e700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fec400a2910 0x7fec400a4dd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:43.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.005+0000 7fec5de8e700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec58103ef0 0x7fec58101a40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:43.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.005+0000 7fec5de8e700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec581004d0 0x7fec58102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:43.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.005+0000 7fec5de8e700 1 --2- 172.21.15.17:0/2051734786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec58103550 0x7fec58101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:43.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.005+0000 7fec5de8e700 1 -- 172.21.15.17:0/2051734786 >> 172.21.15.17:0/2051734786 conn(0x7fec580fbe30 msgr2=0x7fec58106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:43.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.005+0000 7fec5de8e700 1 -- 172.21.15.17:0/2051734786 shutdown_connections 2024-03-24T02:57:43.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:43.005+0000 7fec5de8e700 1 -- 172.21.15.17:0/2051734786 wait complete. 2024-03-24T02:57:43.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 02:57:42 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T02:57:42.732+0000 7faeab74f700 -1 osd.11 0 waiting for initial osdmap 2024-03-24T02:57:43.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 02:57:42 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T02:57:42.748+0000 7faea353a700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T02:57:43.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:43 smithi188 ceph-mon[36206]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:43.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:43 smithi188 ceph-mon[36206]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-03-24T02:57:43.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:43 smithi188 ceph-mon[36206]: osdmap e79: 12 total, 11 up, 12 in 2024-03-24T02:57:43.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:43 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:43.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:43 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:43.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:43 smithi188 ceph-mon[36206]: from='osd.11 ' entity='osd.11' 2024-03-24T02:57:43.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:43 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2051734786' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T02:57:43.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:43 smithi019 ceph-mon[35944]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:43.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:43 smithi019 ceph-mon[35944]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-03-24T02:57:43.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:43 smithi019 ceph-mon[35944]: osdmap e79: 12 total, 11 up, 12 in 2024-03-24T02:57:43.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:43 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:43.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:43 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:43.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:43 smithi019 ceph-mon[35944]: from='osd.11 ' entity='osd.11' 2024-03-24T02:57:43.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:43 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2051734786' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T02:57:43.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:43 smithi017 ceph-mon[31926]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:43.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:43 smithi017 ceph-mon[31926]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-03-24T02:57:43.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:43 smithi017 ceph-mon[31926]: osdmap e79: 12 total, 11 up, 12 in 2024-03-24T02:57:43.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:43 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:43.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:43 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:43.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:43 smithi017 ceph-mon[31926]: from='osd.11 ' entity='osd.11' 2024-03-24T02:57:43.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:43 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2051734786' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T02:57:43.849 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1711249036,"num_in_osds":12,"osd_in_since":1711249042,"num_remapped_pgs":0} 2024-03-24T02:57:44.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:44 smithi017 ceph-mon[31926]: purged_snaps scrub starts 2024-03-24T02:57:44.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:44 smithi017 ceph-mon[31926]: purged_snaps scrub ok 2024-03-24T02:57:44.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:44.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:44 smithi017 ceph-mon[31926]: osd.11 [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713] boot 2024-03-24T02:57:44.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:44 smithi017 ceph-mon[31926]: osdmap e80: 12 total, 12 up, 12 in 2024-03-24T02:57:44.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:44 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:44.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:44 smithi188 ceph-mon[36206]: purged_snaps scrub starts 2024-03-24T02:57:44.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:44 smithi188 ceph-mon[36206]: purged_snaps scrub ok 2024-03-24T02:57:44.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:44.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:44 smithi188 ceph-mon[36206]: osd.11 [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713] boot 2024-03-24T02:57:44.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:44 smithi188 ceph-mon[36206]: osdmap e80: 12 total, 12 up, 12 in 2024-03-24T02:57:44.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:44 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:44 smithi019 ceph-mon[35944]: purged_snaps scrub starts 2024-03-24T02:57:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:44 smithi019 ceph-mon[35944]: purged_snaps scrub ok 2024-03-24T02:57:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:44 smithi019 ceph-mon[35944]: osd.11 [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713] boot 2024-03-24T02:57:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:44 smithi019 ceph-mon[35944]: osdmap e80: 12 total, 12 up, 12 in 2024-03-24T02:57:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:44 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T02:57:44.850 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-24T02:57:45.095 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:57:45.545 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:45.546 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: Detected new or changed devices on smithi188 2024-03-24T02:57:45.546 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.546 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.546 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:45.546 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:45.546 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:45.546 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:45.546 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi188 to 4270M 2024-03-24T02:57:45.546 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.546 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:45.546 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:45.546 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.547 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:45.547 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:45.547 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:45.547 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.547 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:45 smithi017 ceph-mon[31926]: osdmap e81: 12 total, 12 up, 12 in 2024-03-24T02:57:45.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:45.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: Detected new or changed devices on smithi188 2024-03-24T02:57:45.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:45.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:45.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:45.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:45.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi188 to 4270M 2024-03-24T02:57:45.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:45.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:45.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:45.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:45.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:45.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:45 smithi188 ceph-mon[36206]: osdmap e81: 12 total, 12 up, 12 in 2024-03-24T02:57:45.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-24T02:57:45.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: Detected new or changed devices on smithi188 2024-03-24T02:57:45.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:45.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:45.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-24T02:57:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi188 to 4270M 2024-03-24T02:57:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:45.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:45 smithi019 ceph-mon[35944]: osdmap e81: 12 total, 12 up, 12 in 2024-03-24T02:57:46.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.842+0000 7fa607c95700 1 -- 172.21.15.17:0/1200082076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa600103d60 msgr2=0x7fa60010a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:46.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.842+0000 7fa607c95700 1 --2- 172.21.15.17:0/1200082076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa600103d60 0x7fa60010a610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800b0d0 tx=0x7fa5f800b3e0 comp rx=0 tx=0).stop 2024-03-24T02:57:46.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.842+0000 7fa607c95700 1 -- 172.21.15.17:0/1200082076 shutdown_connections 2024-03-24T02:57:46.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.842+0000 7fa607c95700 1 --2- 172.21.15.17:0/1200082076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa600103d60 0x7fa60010a610 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:46.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.842+0000 7fa607c95700 1 --2- 172.21.15.17:0/1200082076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6001021c0 0x7fa6001025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:46.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.842+0000 7fa607c95700 1 --2- 172.21.15.17:0/1200082076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6001033c0 0x7fa600103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:46.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.842+0000 7fa607c95700 1 -- 172.21.15.17:0/1200082076 >> 172.21.15.17:0/1200082076 conn(0x7fa6000fd7a0 msgr2=0x7fa6000ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:46.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.842+0000 7fa607c95700 1 -- 172.21.15.17:0/1200082076 shutdown_connections 2024-03-24T02:57:46.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.842+0000 7fa607c95700 1 -- 172.21.15.17:0/1200082076 wait complete. 2024-03-24T02:57:46.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.843+0000 7fa607c95700 1 Processor -- start 2024-03-24T02:57:46.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.843+0000 7fa607c95700 1 -- start start 2024-03-24T02:57:46.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.843+0000 7fa607c95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6001021c0 0x7fa600198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.843+0000 7fa607c95700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6001033c0 0x7fa6001988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.843+0000 7fa607c95700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa600103d60 0x7fa60019f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.843+0000 7fa607c95700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60010ddf0 con 0x7fa600103d60 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.843+0000 7fa607c95700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60010dc70 con 0x7fa6001021c0 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.843+0000 7fa607c95700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60010df70 con 0x7fa6001033c0 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.843+0000 7fa606232700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa600103d60 0x7fa60019f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.843+0000 7fa605a31700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6001021c0 0x7fa600198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.843+0000 7fa606232700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa600103d60 0x7fa60019f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47190/0 (socket says 172.21.15.17:47190) 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa606232700 1 -- 172.21.15.17:0/1924180685 learned_addr learned my addr 172.21.15.17:0/1924180685 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa605230700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6001033c0 0x7fa6001988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa606232700 1 -- 172.21.15.17:0/1924180685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6001033c0 msgr2=0x7fa6001988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa606232700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6001033c0 0x7fa6001988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa606232700 1 -- 172.21.15.17:0/1924180685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6001021c0 msgr2=0x7fa600198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:46.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa606232700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6001021c0 0x7fa600198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:46.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa606232700 1 -- 172.21.15.17:0/1924180685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60019ffd0 con 0x7fa600103d60 2024-03-24T02:57:46.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa606232700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa600103d60 0x7fa60019f930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8007d10 tx=0x7fa5f8007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:46.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa5f6ffd700 1 -- 172.21.15.17:0/1924180685 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f801f020 con 0x7fa600103d60 2024-03-24T02:57:46.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa5f6ffd700 1 -- 172.21.15.17:0/1924180685 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5f8003e10 con 0x7fa600103d60 2024-03-24T02:57:46.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa607c95700 1 -- 172.21.15.17:0/1924180685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6001a0260 con 0x7fa600103d60 2024-03-24T02:57:46.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa5f6ffd700 1 -- 172.21.15.17:0/1924180685 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f801b750 con 0x7fa600103d60 2024-03-24T02:57:46.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.844+0000 7fa607c95700 1 -- 172.21.15.17:0/1924180685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6001a0790 con 0x7fa600103d60 2024-03-24T02:57:46.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.845+0000 7fa607c95700 1 -- 172.21.15.17:0/1924180685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa600068e50 con 0x7fa600103d60 2024-03-24T02:57:46.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.846+0000 7fa5f6ffd700 1 -- 172.21.15.17:0/1924180685 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa5f8022020 con 0x7fa600103d60 2024-03-24T02:57:46.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.847+0000 7fa5f6ffd700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa5ec0a2910 0x7fa5ec0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:46.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.847+0000 7fa605a31700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa5ec0a2910 0x7fa5ec0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:46.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.847+0000 7fa5f6ffd700 1 -- 172.21.15.17:0/1924180685 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa5f800f020 con 0x7fa600103d60 2024-03-24T02:57:46.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.847+0000 7fa605a31700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa5ec0a2910 0x7fa5ec0a4dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa600103220 tx=0x7fa5f000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:46.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:46.851+0000 7fa5f6ffd700 1 -- 172.21.15.17:0/1924180685 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa5f809d190 con 0x7fa600103d60 2024-03-24T02:57:47.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.053+0000 7fa607c95700 1 -- 172.21.15.17:0/1924180685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fa60002da30 con 0x7fa600103d60 2024-03-24T02:57:47.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.053+0000 7fa5f6ffd700 1 -- 172.21.15.17:0/1924180685 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7fa5f809ca90 con 0x7fa600103d60 2024-03-24T02:57:47.053 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:57:47.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.055+0000 7fa607c95700 1 -- 172.21.15.17:0/1924180685 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa5ec0a2910 msgr2=0x7fa5ec0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:47.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.055+0000 7fa607c95700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa5ec0a2910 0x7fa5ec0a4dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa600103220 tx=0x7fa5f000a380 comp rx=0 tx=0).stop 2024-03-24T02:57:47.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.055+0000 7fa607c95700 1 -- 172.21.15.17:0/1924180685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa600103d60 msgr2=0x7fa60019f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:47.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.055+0000 7fa607c95700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa600103d60 0x7fa60019f930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8007d10 tx=0x7fa5f8007d40 comp rx=0 tx=0).stop 2024-03-24T02:57:47.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.055+0000 7fa607c95700 1 -- 172.21.15.17:0/1924180685 shutdown_connections 2024-03-24T02:57:47.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.055+0000 7fa607c95700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa5ec0a2910 0x7fa5ec0a4dd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:47.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.055+0000 7fa607c95700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa600103d60 0x7fa60019f930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:47.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.055+0000 7fa607c95700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6001021c0 0x7fa600198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:47.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.055+0000 7fa607c95700 1 --2- 172.21.15.17:0/1924180685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6001033c0 0x7fa6001988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:47.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.055+0000 7fa607c95700 1 -- 172.21.15.17:0/1924180685 >> 172.21.15.17:0/1924180685 conn(0x7fa6000fd7a0 msgr2=0x7fa600106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:47.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.055+0000 7fa607c95700 1 -- 172.21.15.17:0/1924180685 shutdown_connections 2024-03-24T02:57:47.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:47.055+0000 7fa607c95700 1 -- 172.21.15.17:0/1924180685 wait complete. 2024-03-24T02:57:47.675 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:47 smithi017 ceph-mon[31926]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:47.675 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:47 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1924180685' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T02:57:47.706 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1711249063,"num_in_osds":12,"osd_in_since":1711249042,"num_remapped_pgs":0} 2024-03-24T02:57:47.706 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T02:57:47.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:47 smithi188 ceph-mon[36206]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:47.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:47 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1924180685' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T02:57:47.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:47 smithi019 ceph-mon[35944]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:47.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:47 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1924180685' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T02:57:47.951 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:57:49.572 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:49 smithi017 ceph-mon[31926]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.571+0000 7f069103f700 1 -- 172.21.15.17:0/3757239538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c06a8e0 msgr2=0x7f068c06ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.571+0000 7f069103f700 1 --2- 172.21.15.17:0/3757239538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c06a8e0 0x7f068c06ad60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f067c0099c0 tx=0x7f067c009cd0 comp rx=0 tx=0).stop 2024-03-24T02:57:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.572+0000 7f069103f700 1 -- 172.21.15.17:0/3757239538 shutdown_connections 2024-03-24T02:57:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.572+0000 7f069103f700 1 --2- 172.21.15.17:0/3757239538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f068c06b2a0 0x7f068c10efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.572+0000 7f069103f700 1 --2- 172.21.15.17:0/3757239538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c1081c0 0x7f068c1085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.572+0000 7f069103f700 1 --2- 172.21.15.17:0/3757239538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c06a8e0 0x7f068c06ad60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.572+0000 7f069103f700 1 -- 172.21.15.17:0/3757239538 >> 172.21.15.17:0/3757239538 conn(0x7f068c0780a0 msgr2=0x7f068c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.572+0000 7f069103f700 1 -- 172.21.15.17:0/3757239538 shutdown_connections 2024-03-24T02:57:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.572+0000 7f069103f700 1 -- 172.21.15.17:0/3757239538 wait complete. 2024-03-24T02:57:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f069103f700 1 Processor -- start 2024-03-24T02:57:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f069103f700 1 -- start start 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f069103f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f068c06a8e0 0x7f068c101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f069103f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c06b2a0 0x7f068c101680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f069103f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c1081c0 0x7f068c101bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f069103f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c112470 con 0x7f068c1081c0 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f069103f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c1122f0 con 0x7f068c06b2a0 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f069103f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c1125f0 con 0x7f068c06a8e0 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f068b59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c1081c0 0x7f068c101bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f068b59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c1081c0 0x7f068c101bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47208/0 (socket says 172.21.15.17:47208) 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f068b59e700 1 -- 172.21.15.17:0/3451315295 learned_addr learned my addr 172.21.15.17:0/3451315295 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f068ad9d700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f068c06a8e0 0x7f068c101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f068a59c700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c06b2a0 0x7f068c101680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f068b59e700 1 -- 172.21.15.17:0/3451315295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f068c06a8e0 msgr2=0x7f068c101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f068b59e700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f068c06a8e0 0x7f068c101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f068b59e700 1 -- 172.21.15.17:0/3451315295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c06b2a0 msgr2=0x7f068c101680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f068b59e700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c06b2a0 0x7f068c101680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.573+0000 7f068b59e700 1 -- 172.21.15.17:0/3451315295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c1a6d80 con 0x7f068c1081c0 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.574+0000 7f068b59e700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c1081c0 0x7f068c101bc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0684007ee0 tx=0x7f068400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.574+0000 7f0673fff700 1 -- 172.21.15.17:0/3451315295 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0684004bb0 con 0x7f068c1081c0 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.574+0000 7f069103f700 1 -- 172.21.15.17:0/3451315295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c1a7070 con 0x7f068c1081c0 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.574+0000 7f0673fff700 1 -- 172.21.15.17:0/3451315295 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0684004d50 con 0x7f068c1081c0 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.574+0000 7f0673fff700 1 -- 172.21.15.17:0/3451315295 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0684005960 con 0x7f068c1081c0 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.574+0000 7f069103f700 1 -- 172.21.15.17:0/3451315295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068c1a75d0 con 0x7f068c1081c0 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.575+0000 7f069103f700 1 -- 172.21.15.17:0/3451315295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068c068e50 con 0x7f068c1081c0 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.576+0000 7f0673fff700 1 -- 172.21.15.17:0/3451315295 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f068401a020 con 0x7f068c1081c0 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.576+0000 7f0673fff700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f06740a2910 0x7f06740a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.577+0000 7f068ad9d700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f06740a2910 0x7f06740a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.577+0000 7f0673fff700 1 -- 172.21.15.17:0/3451315295 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06840cdd90 con 0x7f068c1081c0 2024-03-24T02:57:49.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.577+0000 7f068ad9d700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f06740a2910 0x7f06740a4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0680005950 tx=0x7f068000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:49.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.581+0000 7f0673fff700 1 -- 172.21.15.17:0/3451315295 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0684010070 con 0x7f068c1081c0 2024-03-24T02:57:49.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.787+0000 7f069103f700 1 -- 172.21.15.17:0/3451315295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f068c02dbf0 con 0x7f068c1081c0 2024-03-24T02:57:49.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.788+0000 7f0673fff700 1 -- 172.21.15.17:0/3451315295 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18428 (secure 0 0 0) 0x7f068409b230 con 0x7f068c1081c0 2024-03-24T02:57:49.789 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:57:49.789 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":81,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:57:45.021539+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:57:49.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.790+0000 7f069103f700 1 -- 172.21.15.17:0/3451315295 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f06740a2910 msgr2=0x7f06740a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:49.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.790+0000 7f069103f700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f06740a2910 0x7f06740a4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0680005950 tx=0x7f068000b410 comp rx=0 tx=0).stop 2024-03-24T02:57:49.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.790+0000 7f069103f700 1 -- 172.21.15.17:0/3451315295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c1081c0 msgr2=0x7f068c101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:49.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.790+0000 7f069103f700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c1081c0 0x7f068c101bc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0684007ee0 tx=0x7f068400d3b0 comp rx=0 tx=0).stop 2024-03-24T02:57:49.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.790+0000 7f069103f700 1 -- 172.21.15.17:0/3451315295 shutdown_connections 2024-03-24T02:57:49.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.790+0000 7f069103f700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f06740a2910 0x7f06740a4dd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:49.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.790+0000 7f069103f700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c1081c0 0x7f068c101bc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:49.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.790+0000 7f069103f700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f068c06a8e0 0x7f068c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:49.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.790+0000 7f069103f700 1 --2- 172.21.15.17:0/3451315295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c06b2a0 0x7f068c101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:49.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.790+0000 7f069103f700 1 -- 172.21.15.17:0/3451315295 >> 172.21.15.17:0/3451315295 conn(0x7f068c0780a0 msgr2=0x7f068c0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:49.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.790+0000 7f069103f700 1 -- 172.21.15.17:0/3451315295 shutdown_connections 2024-03-24T02:57:49.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:49.790+0000 7f069103f700 1 -- 172.21.15.17:0/3451315295 wait complete. 2024-03-24T02:57:49.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:49 smithi188 ceph-mon[36206]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:49.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:49 smithi019 ceph-mon[35944]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:50.556 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T02:53:54.207139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-24T02:57:50.556 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-24T02:57:50.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:50 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3451315295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:57:50.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:50 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3451315295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:57:50.800 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:57:50.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:50 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3451315295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:57:51.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:51 smithi017 ceph-mon[31926]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:51.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:51 smithi188 ceph-mon[36206]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:51.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:51 smithi019 ceph-mon[35944]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:52.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.577+0000 7fc6d3e36700 1 -- 172.21.15.17:0/2010512100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6cc102ba0 msgr2=0x7fc6cc10efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:52.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.577+0000 7fc6d3e36700 1 --2- 172.21.15.17:0/2010512100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6cc102ba0 0x7fc6cc10efe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400b0d0 tx=0x7fc6c400b3e0 comp rx=0 tx=0).stop 2024-03-24T02:57:52.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.577+0000 7fc6d3e36700 1 -- 172.21.15.17:0/2010512100 shutdown_connections 2024-03-24T02:57:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.577+0000 7fc6d3e36700 1 --2- 172.21.15.17:0/2010512100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6cc102ba0 0x7fc6cc10efe0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.577+0000 7fc6d3e36700 1 --2- 172.21.15.17:0/2010512100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6cc1081c0 0x7fc6cc1085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.577+0000 7fc6d3e36700 1 --2- 172.21.15.17:0/2010512100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6cc1021e0 0x7fc6cc102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.577+0000 7fc6d3e36700 1 -- 172.21.15.17:0/2010512100 >> 172.21.15.17:0/2010512100 conn(0x7fc6cc0792c0 msgr2=0x7fc6cc0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.577+0000 7fc6d3e36700 1 -- 172.21.15.17:0/2010512100 shutdown_connections 2024-03-24T02:57:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.577+0000 7fc6d3e36700 1 -- 172.21.15.17:0/2010512100 wait complete. 2024-03-24T02:57:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.577+0000 7fc6d3e36700 1 Processor -- start 2024-03-24T02:57:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.577+0000 7fc6d3e36700 1 -- start start 2024-03-24T02:57:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d3e36700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6cc1021e0 0x7fc6cc074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d3e36700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6cc102ba0 0x7fc6cc0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d3e36700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6cc1081c0 0x7fc6cc074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d3e36700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc1124e0 con 0x7fc6cc1021e0 2024-03-24T02:57:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d3e36700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc112360 con 0x7fc6cc1081c0 2024-03-24T02:57:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d3e36700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc112660 con 0x7fc6cc102ba0 2024-03-24T02:57:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d1bd2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6cc1021e0 0x7fc6cc074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d1bd2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6cc1021e0 0x7fc6cc074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51692/0 (socket says 172.21.15.17:51692) 2024-03-24T02:57:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d1bd2700 1 -- 172.21.15.17:0/1783908008 learned_addr learned my addr 172.21.15.17:0/1783908008 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:57:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d13d1700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6cc102ba0 0x7fc6cc0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d23d3700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6cc1081c0 0x7fc6cc074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d1bd2700 1 -- 172.21.15.17:0/1783908008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6cc102ba0 msgr2=0x7fc6cc0749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d1bd2700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6cc102ba0 0x7fc6cc0749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d1bd2700 1 -- 172.21.15.17:0/1783908008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6cc1081c0 msgr2=0x7fc6cc074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d1bd2700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6cc1081c0 0x7fc6cc074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d1bd2700 1 -- 172.21.15.17:0/1783908008 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc1a6de0 con 0x7fc6cc1021e0 2024-03-24T02:57:52.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d23d3700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6cc1081c0 0x7fc6cc074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:57:52.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d1bd2700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6cc1021e0 0x7fc6cc074490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800d980 tx=0x7fc6c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:52.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6c2ffd700 1 -- 172.21.15.17:0/1783908008 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8017070 con 0x7fc6cc1021e0 2024-03-24T02:57:52.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6c2ffd700 1 -- 172.21.15.17:0/1783908008 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6c8005020 con 0x7fc6cc1021e0 2024-03-24T02:57:52.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6c2ffd700 1 -- 172.21.15.17:0/1783908008 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8009bc0 con 0x7fc6cc1021e0 2024-03-24T02:57:52.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.578+0000 7fc6d3e36700 1 -- 172.21.15.17:0/1783908008 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc1a70d0 con 0x7fc6cc1021e0 2024-03-24T02:57:52.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.579+0000 7fc6d3e36700 1 -- 172.21.15.17:0/1783908008 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc1a7610 con 0x7fc6cc1021e0 2024-03-24T02:57:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.579+0000 7fc6d3e36700 1 -- 172.21.15.17:0/1783908008 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc068e50 con 0x7fc6cc1021e0 2024-03-24T02:57:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.580+0000 7fc6c2ffd700 1 -- 172.21.15.17:0/1783908008 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc6c80051d0 con 0x7fc6cc1021e0 2024-03-24T02:57:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.581+0000 7fc6c2ffd700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc6b80a2910 0x7fc6b80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.581+0000 7fc6d13d1700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc6b80a2910 0x7fc6b80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.582+0000 7fc6c2ffd700 1 -- 172.21.15.17:0/1783908008 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc6c80ce910 con 0x7fc6cc1021e0 2024-03-24T02:57:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.582+0000 7fc6d13d1700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc6b80a2910 0x7fc6b80a4dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc079850 tx=0x7fc6bc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:52.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.585+0000 7fc6c2ffd700 1 -- 172.21.15.17:0/1783908008 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc6c8020020 con 0x7fc6cc1021e0 2024-03-24T02:57:52.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.787+0000 7fc6d3e36700 1 -- 172.21.15.17:0/1783908008 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6cc02da30 con 0x7fc6cc1021e0 2024-03-24T02:57:52.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.787+0000 7fc6c2ffd700 1 -- 172.21.15.17:0/1783908008 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6c809be30 con 0x7fc6cc1021e0 2024-03-24T02:57:52.788 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-24T02:57:52.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.789+0000 7fc6d3e36700 1 -- 172.21.15.17:0/1783908008 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc6b80a2910 msgr2=0x7fc6b80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:52.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.789+0000 7fc6d3e36700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc6b80a2910 0x7fc6b80a4dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc079850 tx=0x7fc6bc009190 comp rx=0 tx=0).stop 2024-03-24T02:57:52.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.789+0000 7fc6d3e36700 1 -- 172.21.15.17:0/1783908008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6cc1021e0 msgr2=0x7fc6cc074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:52.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.789+0000 7fc6d3e36700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6cc1021e0 0x7fc6cc074490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800d980 tx=0x7fc6c800dc90 comp rx=0 tx=0).stop 2024-03-24T02:57:52.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.789+0000 7fc6d3e36700 1 -- 172.21.15.17:0/1783908008 shutdown_connections 2024-03-24T02:57:52.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.789+0000 7fc6d3e36700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc6b80a2910 0x7fc6b80a4dd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:52.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.789+0000 7fc6d3e36700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6cc1081c0 0x7fc6cc074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:52.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.789+0000 7fc6d3e36700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6cc1021e0 0x7fc6cc074490 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:52.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.789+0000 7fc6d3e36700 1 --2- 172.21.15.17:0/1783908008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6cc102ba0 0x7fc6cc0749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:52.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.789+0000 7fc6d3e36700 1 -- 172.21.15.17:0/1783908008 >> 172.21.15.17:0/1783908008 conn(0x7fc6cc0792c0 msgr2=0x7fc6cc100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:52.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.790+0000 7fc6d3e36700 1 -- 172.21.15.17:0/1783908008 shutdown_connections 2024-03-24T02:57:52.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:52.790+0000 7fc6d3e36700 1 -- 172.21.15.17:0/1783908008 wait complete. 2024-03-24T02:57:53.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:53 smithi017 ceph-mon[31926]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:53.535 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:53 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1783908008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:57:53.564 INFO:tasks.cephadm:Adding mds.a on smithi017 2024-03-24T02:57:53.565 INFO:tasks.cephadm:Adding mds.d on smithi017 2024-03-24T02:57:53.565 INFO:tasks.cephadm:Adding mds.g on smithi017 2024-03-24T02:57:53.565 INFO:tasks.cephadm:Adding mds.j on smithi017 2024-03-24T02:57:53.565 INFO:tasks.cephadm:Adding mds.b on smithi019 2024-03-24T02:57:53.565 INFO:tasks.cephadm:Adding mds.e on smithi019 2024-03-24T02:57:53.565 INFO:tasks.cephadm:Adding mds.h on smithi019 2024-03-24T02:57:53.565 INFO:tasks.cephadm:Adding mds.k on smithi019 2024-03-24T02:57:53.565 INFO:tasks.cephadm:Adding mds.c on smithi188 2024-03-24T02:57:53.565 INFO:tasks.cephadm:Adding mds.f on smithi188 2024-03-24T02:57:53.565 INFO:tasks.cephadm:Adding mds.i on smithi188 2024-03-24T02:57:53.565 INFO:tasks.cephadm:Adding mds.l on smithi188 2024-03-24T02:57:53.565 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph orch apply mds all '12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l' 2024-03-24T02:57:53.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:53 smithi188 ceph-mon[36206]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:53.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:53 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1783908008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:57:53.816 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T02:57:53.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:53 smithi019 ceph-mon[35944]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:53.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:53 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1783908008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:57:55.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.412+0000 7fab7446f700 1 -- 172.21.15.188:0/427988075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab6c1033f0 msgr2=0x7fab6c103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:55.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.412+0000 7fab7446f700 1 --2- 172.21.15.188:0/427988075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab6c1033f0 0x7fab6c103850 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009990 tx=0x7fab5c009ca0 comp rx=0 tx=0).stop 2024-03-24T02:57:55.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.412+0000 7fab7446f700 1 -- 172.21.15.188:0/427988075 shutdown_connections 2024-03-24T02:57:55.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.412+0000 7fab7446f700 1 --2- 172.21.15.188:0/427988075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c103d90 0x7fab6c10a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:55.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.412+0000 7fab7446f700 1 --2- 172.21.15.188:0/427988075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab6c1021f0 0x7fab6c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:55.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.412+0000 7fab7446f700 1 --2- 172.21.15.188:0/427988075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab6c1033f0 0x7fab6c103850 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:55.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.412+0000 7fab7446f700 1 -- 172.21.15.188:0/427988075 >> 172.21.15.188:0/427988075 conn(0x7fab6c0fd790 msgr2=0x7fab6c0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:55.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.412+0000 7fab7446f700 1 -- 172.21.15.188:0/427988075 shutdown_connections 2024-03-24T02:57:55.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.412+0000 7fab7446f700 1 -- 172.21.15.188:0/427988075 wait complete. 2024-03-24T02:57:55.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab7446f700 1 Processor -- start 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab7446f700 1 -- start start 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab7446f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab6c1021f0 0x7fab6c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab7446f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1033f0 0x7fab6c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab7446f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab6c103d90 0x7fab6c1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab7446f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c10ddb0 con 0x7fab6c1033f0 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab7446f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c10dc30 con 0x7fab6c1021f0 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab7446f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c10df30 con 0x7fab6c103d90 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab72a0c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab6c103d90 0x7fab6c1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab72a0c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab6c103d90 0x7fab6c1a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:44522/0 (socket says 172.21.15.188:44522) 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab72a0c700 1 -- 172.21.15.188:0/1706812284 learned_addr learned my addr 172.21.15.188:0/1706812284 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab71a0a700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1033f0 0x7fab6c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab7220b700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab6c1021f0 0x7fab6c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab72a0c700 1 -- 172.21.15.188:0/1706812284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab6c1021f0 msgr2=0x7fab6c19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:55.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab72a0c700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab6c1021f0 0x7fab6c19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:55.416 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab72a0c700 1 -- 172.21.15.188:0/1706812284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1033f0 msgr2=0x7fab6c19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:55.416 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab72a0c700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1033f0 0x7fab6c19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:55.416 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab72a0c700 1 -- 172.21.15.188:0/1706812284 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c1a2170 con 0x7fab6c103d90 2024-03-24T02:57:55.416 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab71a0a700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1033f0 0x7fab6c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:57:55.416 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab72a0c700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab6c103d90 0x7fab6c1a1ad0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fab6400d9b0 tx=0x7fab6400dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:55.416 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.413+0000 7fab637fe700 1 -- 172.21.15.188:0/1706812284 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64017070 con 0x7fab6c103d90 2024-03-24T02:57:55.416 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.414+0000 7fab7446f700 1 -- 172.21.15.188:0/1706812284 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c1a2460 con 0x7fab6c103d90 2024-03-24T02:57:55.416 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.414+0000 7fab637fe700 1 -- 172.21.15.188:0/1706812284 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab64009e20 con 0x7fab6c103d90 2024-03-24T02:57:55.416 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.414+0000 7fab7446f700 1 -- 172.21.15.188:0/1706812284 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c1a29a0 con 0x7fab6c103d90 2024-03-24T02:57:55.416 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.414+0000 7fab637fe700 1 -- 172.21.15.188:0/1706812284 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab640053b0 con 0x7fab6c103d90 2024-03-24T02:57:55.417 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.414+0000 7fab617fa700 1 -- 172.21.15.188:0/1706812284 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab54005330 con 0x7fab6c103d90 2024-03-24T02:57:55.417 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.416+0000 7fab637fe700 1 -- 172.21.15.188:0/1706812284 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fab64004750 con 0x7fab6c103d90 2024-03-24T02:57:55.417 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.416+0000 7fab637fe700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fab580a2890 0x7fab580a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:55.417 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.416+0000 7fab637fe700 1 -- 172.21.15.188:0/1706812284 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fab640ceaa0 con 0x7fab6c103d90 2024-03-24T02:57:55.417 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.416+0000 7fab7220b700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fab580a2890 0x7fab580a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:55.417 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.417+0000 7fab7220b700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fab580a2890 0x7fab580a4d50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab6c103250 tx=0x7fab68009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:55.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.420+0000 7fab637fe700 1 -- 172.21.15.188:0/1706812284 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab64020020 con 0x7fab6c103d90 2024-03-24T02:57:55.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:55 smithi188 ceph-mon[36206]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:55.627 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.627+0000 7fab617fa700 1 -- 172.21.15.188:0/1706812284 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fab54000cc0 con 0x7fab580a2890 2024-03-24T02:57:55.635 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.635+0000 7fab637fe700 1 -- 172.21.15.188:0/1706812284 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fab54000cc0 con 0x7fab580a2890 2024-03-24T02:57:55.635 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mds.all update... 2024-03-24T02:57:55.637 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.638+0000 7fab617fa700 1 -- 172.21.15.188:0/1706812284 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fab580a2890 msgr2=0x7fab580a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:55.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.638+0000 7fab617fa700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fab580a2890 0x7fab580a4d50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab6c103250 tx=0x7fab68009040 comp rx=0 tx=0).stop 2024-03-24T02:57:55.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.638+0000 7fab617fa700 1 -- 172.21.15.188:0/1706812284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab6c103d90 msgr2=0x7fab6c1a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:55.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.638+0000 7fab617fa700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab6c103d90 0x7fab6c1a1ad0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fab6400d9b0 tx=0x7fab6400dd70 comp rx=0 tx=0).stop 2024-03-24T02:57:55.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.638+0000 7fab617fa700 1 -- 172.21.15.188:0/1706812284 shutdown_connections 2024-03-24T02:57:55.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.638+0000 7fab617fa700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fab580a2890 0x7fab580a4d50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:55.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.638+0000 7fab617fa700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab6c103d90 0x7fab6c1a1ad0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:55.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.638+0000 7fab617fa700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab6c1021f0 0x7fab6c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:55.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.638+0000 7fab617fa700 1 --2- 172.21.15.188:0/1706812284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1033f0 0x7fab6c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:55.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.638+0000 7fab617fa700 1 -- 172.21.15.188:0/1706812284 >> 172.21.15.188:0/1706812284 conn(0x7fab6c0fd790 msgr2=0x7fab6c0743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:55.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.638+0000 7fab617fa700 1 -- 172.21.15.188:0/1706812284 shutdown_connections 2024-03-24T02:57:55.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-24T02:57:55.638+0000 7fab617fa700 1 -- 172.21.15.188:0/1706812284 wait complete. 2024-03-24T02:57:55.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:55 smithi017 ceph-mon[31926]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:55.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:55 smithi019 ceph-mon[35944]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:56.314 DEBUG:teuthology.orchestra.run.smithi017:mds.a> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.a.service 2024-03-24T02:57:56.316 DEBUG:teuthology.orchestra.run.smithi017:mds.d> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.d.service 2024-03-24T02:57:56.360 DEBUG:teuthology.orchestra.run.smithi017:mds.g> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.g.service 2024-03-24T02:57:56.403 DEBUG:teuthology.orchestra.run.smithi017:mds.j> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.j.service 2024-03-24T02:57:56.446 DEBUG:teuthology.orchestra.run.smithi019:mds.b> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.b.service 2024-03-24T02:57:56.448 DEBUG:teuthology.orchestra.run.smithi019:mds.e> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.e.service 2024-03-24T02:57:56.491 DEBUG:teuthology.orchestra.run.smithi019:mds.h> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.h.service 2024-03-24T02:57:56.534 DEBUG:teuthology.orchestra.run.smithi019:mds.k> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.k.service 2024-03-24T02:57:56.577 DEBUG:teuthology.orchestra.run.smithi188:mds.c> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.c.service 2024-03-24T02:57:56.579 DEBUG:teuthology.orchestra.run.smithi188:mds.f> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.f.service 2024-03-24T02:57:56.622 DEBUG:teuthology.orchestra.run.smithi188:mds.i> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.i.service 2024-03-24T02:57:56.665 DEBUG:teuthology.orchestra.run.smithi188:mds.l> sudo journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.l.service 2024-03-24T02:57:56.708 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-24T02:57:56.708 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T02:57:56.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:56 smithi019 ceph-mon[35944]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:57:56.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:56 smithi019 ceph-mon[35944]: Saving service mds.all spec with placement smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-03-24T02:57:56.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:56 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:56.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:56 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:56.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:56 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:56.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:56 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:56.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:56 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:56.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:56 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:57:56.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:56 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:57:56.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:56 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.829+0000 7fcc79760640 1 -- 172.21.15.17:0/2105481034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc741016d0 msgr2=0x7fcc74101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.829+0000 7fcc79760640 1 --2- 172.21.15.17:0/2105481034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc741016d0 0x7fcc74101af0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009900 tx=0x7fcc6402bc30 comp rx=0 tx=0).stop 2024-03-24T02:57:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.829+0000 7fcc79760640 1 -- 172.21.15.17:0/2105481034 shutdown_connections 2024-03-24T02:57:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.829+0000 7fcc79760640 1 --2- 172.21.15.17:0/2105481034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc74103280 0x7fcc74109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.829+0000 7fcc79760640 1 --2- 172.21.15.17:0/2105481034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc741028c0 0x7fcc74102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.829+0000 7fcc79760640 1 --2- 172.21.15.17:0/2105481034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc741016d0 0x7fcc74101af0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.829+0000 7fcc79760640 1 -- 172.21.15.17:0/2105481034 >> 172.21.15.17:0/2105481034 conn(0x7fcc740fcc80 msgr2=0x7fcc740ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.829+0000 7fcc79760640 1 -- 172.21.15.17:0/2105481034 shutdown_connections 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.829+0000 7fcc79760640 1 -- 172.21.15.17:0/2105481034 wait complete. 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.829+0000 7fcc79760640 1 Processor -- start 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc79760640 1 -- start start 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc79760640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc741016d0 0x7fcc741979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc79760640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc741028c0 0x7fcc74197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc79760640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc74103280 0x7fcc7419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc79760640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7410d350 con 0x7fcc741016d0 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc79760640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7410d1d0 con 0x7fcc74103280 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc79760640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7410d4d0 con 0x7fcc741028c0 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc72ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc741016d0 0x7fcc741979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc72ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc741016d0 0x7fcc741979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51712/0 (socket says 172.21.15.17:51712) 2024-03-24T02:57:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc72ffd640 1 -- 172.21.15.17:0/1582742436 learned_addr learned my addr 172.21.15.17:0/1582742436 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:57:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc727fc640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc741028c0 0x7fcc74197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc737fe640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc74103280 0x7fcc7419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc72ffd640 1 -- 172.21.15.17:0/1582742436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc741028c0 msgr2=0x7fcc74197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc72ffd640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc741028c0 0x7fcc74197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc72ffd640 1 -- 172.21.15.17:0/1582742436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc74103280 msgr2=0x7fcc7419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc72ffd640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc74103280 0x7fcc7419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc72ffd640 1 -- 172.21.15.17:0/1582742436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7419f5e0 con 0x7fcc741016d0 2024-03-24T02:57:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc737fe640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc74103280 0x7fcc7419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:57:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.830+0000 7fcc72ffd640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc741016d0 0x7fcc741979c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005ab0 tx=0x7fcc64005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.831+0000 7fcc5bfff640 1 -- 172.21.15.17:0/1582742436 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64041070 con 0x7fcc741016d0 2024-03-24T02:57:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.831+0000 7fcc5bfff640 1 -- 172.21.15.17:0/1582742436 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc640054e0 con 0x7fcc741016d0 2024-03-24T02:57:56.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.831+0000 7fcc79760640 1 -- 172.21.15.17:0/1582742436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7419f870 con 0x7fcc741016d0 2024-03-24T02:57:56.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.831+0000 7fcc5bfff640 1 -- 172.21.15.17:0/1582742436 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64034440 con 0x7fcc741016d0 2024-03-24T02:57:56.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.831+0000 7fcc79760640 1 -- 172.21.15.17:0/1582742436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7419fd60 con 0x7fcc741016d0 2024-03-24T02:57:56.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.832+0000 7fcc79760640 1 -- 172.21.15.17:0/1582742436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc74108a40 con 0x7fcc741016d0 2024-03-24T02:57:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.833+0000 7fcc5bfff640 1 -- 172.21.15.17:0/1582742436 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fcc64044070 con 0x7fcc741016d0 2024-03-24T02:57:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.833+0000 7fcc5bfff640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcc5c0a2800 0x7fcc5c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.833+0000 7fcc727fc640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcc5c0a2800 0x7fcc5c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.833+0000 7fcc5bfff640 1 -- 172.21.15.17:0/1582742436 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcc640f0230 con 0x7fcc741016d0 2024-03-24T02:57:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.834+0000 7fcc727fc640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcc5c0a2800 0x7fcc5c0a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcc74198ed0 tx=0x7fcc6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:56.838+0000 7fcc5bfff640 1 -- 172.21.15.17:0/1582742436 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc640bd770 con 0x7fcc741016d0 2024-03-24T02:57:56.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:56 smithi017 ceph-mon[31926]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:57:56.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:56 smithi017 ceph-mon[31926]: Saving service mds.all spec with placement smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-03-24T02:57:56.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:56 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:56.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:56 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:56.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:56 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:56.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:56 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:56.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:56 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:56.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:56 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:57:56.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:56 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:57:56.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:56 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:57.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.022+0000 7fcc79760640 1 -- 172.21.15.17:0/1582742436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc7402d290 con 0x7fcc741016d0 2024-03-24T02:57:57.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.023+0000 7fcc5bfff640 1 -- 172.21.15.17:0/1582742436 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18428 (secure 0 0 0) 0x7fcc640bd070 con 0x7fcc741016d0 2024-03-24T02:57:57.023 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:57:57.023 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":81,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:57:45.021539+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:57:57.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.025+0000 7fcc79760640 1 -- 172.21.15.17:0/1582742436 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcc5c0a2800 msgr2=0x7fcc5c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:57.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.025+0000 7fcc79760640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcc5c0a2800 0x7fcc5c0a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcc74198ed0 tx=0x7fcc6800b410 comp rx=0 tx=0).stop 2024-03-24T02:57:57.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.025+0000 7fcc79760640 1 -- 172.21.15.17:0/1582742436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc741016d0 msgr2=0x7fcc741979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:57.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.025+0000 7fcc79760640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc741016d0 0x7fcc741979c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005ab0 tx=0x7fcc64005dc0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.025+0000 7fcc79760640 1 -- 172.21.15.17:0/1582742436 shutdown_connections 2024-03-24T02:57:57.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.025+0000 7fcc79760640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcc5c0a2800 0x7fcc5c0a4cc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.025+0000 7fcc79760640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc74103280 0x7fcc7419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.025+0000 7fcc79760640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc741028c0 0x7fcc74197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.025+0000 7fcc79760640 1 --2- 172.21.15.17:0/1582742436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc741016d0 0x7fcc741979c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.025+0000 7fcc79760640 1 -- 172.21.15.17:0/1582742436 >> 172.21.15.17:0/1582742436 conn(0x7fcc740fcc80 msgr2=0x7fcc74105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:57.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.025+0000 7fcc79760640 1 -- 172.21.15.17:0/1582742436 shutdown_connections 2024-03-24T02:57:57.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.025+0000 7fcc79760640 1 -- 172.21.15.17:0/1582742436 wait complete. 2024-03-24T02:57:57.037 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T02:53:54.207139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-24T02:57:57.037 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T02:57:57.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:56 smithi188 ceph-mon[36206]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:57:57.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:56 smithi188 ceph-mon[36206]: Saving service mds.all spec with placement smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-03-24T02:57:57.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:56 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:57.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:56 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:57:57.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:56 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:57.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:56 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:57:57.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:56 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:57.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:56 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:57:57.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:56 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:57:57.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:56 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.141+0000 7f74c51f4640 1 -- 172.21.15.17:0/1611934958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74c01017a0 msgr2=0x7f74c0101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.141+0000 7f74c51f4640 1 --2- 172.21.15.17:0/1611934958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74c01017a0 0x7f74c0101c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009900 tx=0x7f74b402bc30 comp rx=0 tx=0).stop 2024-03-24T02:57:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.141+0000 7f74c51f4640 1 -- 172.21.15.17:0/1611934958 shutdown_connections 2024-03-24T02:57:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.141+0000 7f74c51f4640 1 --2- 172.21.15.17:0/1611934958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74c0102160 0x7f74c01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.141+0000 7f74c51f4640 1 --2- 172.21.15.17:0/1611934958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74c01017a0 0x7f74c0101c20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.141+0000 7f74c51f4640 1 --2- 172.21.15.17:0/1611934958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c01005b0 0x7f74c01009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.141+0000 7f74c51f4640 1 -- 172.21.15.17:0/1611934958 >> 172.21.15.17:0/1611934958 conn(0x7f74c00fbb40 msgr2=0x7f74c00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.141+0000 7f74c51f4640 1 -- 172.21.15.17:0/1611934958 shutdown_connections 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.141+0000 7f74c51f4640 1 -- 172.21.15.17:0/1611934958 wait complete. 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.142+0000 7f74c51f4640 1 Processor -- start 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.142+0000 7f74c51f4640 1 -- start start 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.142+0000 7f74c51f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c01005b0 0x7f74c006a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.142+0000 7f74c51f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74c01017a0 0x7f74c019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.142+0000 7f74c51f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74c0102160 0x7f74c019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.142+0000 7f74c51f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c010c230 con 0x7f74c0102160 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74c51f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c010c0b0 con 0x7f74c01005b0 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74c51f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c010c3b0 con 0x7f74c01017a0 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74bf59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74c0102160 0x7f74c019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74bf59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74c0102160 0x7f74c019ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51734/0 (socket says 172.21.15.17:51734) 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74bf59e640 1 -- 172.21.15.17:0/3720279979 learned_addr learned my addr 172.21.15.17:0/3720279979 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:57:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74be59c640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74c01017a0 0x7f74c019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74bed9d640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c01005b0 0x7f74c006a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74bf59e640 1 -- 172.21.15.17:0/3720279979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74c01017a0 msgr2=0x7f74c019c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74bf59e640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74c01017a0 0x7f74c019c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74bf59e640 1 -- 172.21.15.17:0/3720279979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c01005b0 msgr2=0x7f74c006a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74bf59e640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c01005b0 0x7f74c006a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74bf59e640 1 -- 172.21.15.17:0/3720279979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c019f5d0 con 0x7f74c0102160 2024-03-24T02:57:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74bed9d640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c01005b0 0x7f74c006a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:57:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74bf59e640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74c0102160 0x7f74c019ecc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74b000eb10 tx=0x7f74b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74a7fff640 1 -- 172.21.15.17:0/3720279979 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0004b20 con 0x7f74c0102160 2024-03-24T02:57:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74c51f4640 1 -- 172.21.15.17:0/3720279979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c019f860 con 0x7f74c0102160 2024-03-24T02:57:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74a7fff640 1 -- 172.21.15.17:0/3720279979 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74b0004cc0 con 0x7f74c0102160 2024-03-24T02:57:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74a7fff640 1 -- 172.21.15.17:0/3720279979 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0010760 con 0x7f74c0102160 2024-03-24T02:57:57.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.143+0000 7f74c51f4640 1 -- 172.21.15.17:0/3720279979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c019fd80 con 0x7f74c0102160 2024-03-24T02:57:57.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.144+0000 7f74c51f4640 1 -- 172.21.15.17:0/3720279979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c00688e0 con 0x7f74c0102160 2024-03-24T02:57:57.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.145+0000 7f74a7fff640 1 -- 172.21.15.17:0/3720279979 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f74b0005020 con 0x7f74c0102160 2024-03-24T02:57:57.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.146+0000 7f74a7fff640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f74a80a2800 0x7f74a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:57.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.146+0000 7f74a7fff640 1 -- 172.21.15.17:0/3720279979 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f74b00ced50 con 0x7f74c0102160 2024-03-24T02:57:57.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.146+0000 7f74bed9d640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f74a80a2800 0x7f74a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:57.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.146+0000 7f74bed9d640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f74a80a2800 0x7f74a80a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f74c0101600 tx=0x7f74b8005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:57.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.150+0000 7f74a7fff640 1 -- 172.21.15.17:0/3720279979 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f74b00d2030 con 0x7f74c0102160 2024-03-24T02:57:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.336+0000 7f74c51f4640 1 -- 172.21.15.17:0/3720279979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74c002d070 con 0x7f74c0102160 2024-03-24T02:57:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.336+0000 7f74a7fff640 1 -- 172.21.15.17:0/3720279979 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74b009c290 con 0x7f74c0102160 2024-03-24T02:57:57.337 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-24T02:57:57.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.338+0000 7f74c51f4640 1 -- 172.21.15.17:0/3720279979 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f74a80a2800 msgr2=0x7f74a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:57.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.338+0000 7f74c51f4640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f74a80a2800 0x7f74a80a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f74c0101600 tx=0x7f74b8005db0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.338+0000 7f74c51f4640 1 -- 172.21.15.17:0/3720279979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74c0102160 msgr2=0x7f74c019ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:57.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.338+0000 7f74c51f4640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74c0102160 0x7f74c019ecc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74b000eb10 tx=0x7f74b000c2d0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.338+0000 7f74c51f4640 1 -- 172.21.15.17:0/3720279979 shutdown_connections 2024-03-24T02:57:57.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.338+0000 7f74c51f4640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f74a80a2800 0x7f74a80a4cc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.338+0000 7f74c51f4640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74c0102160 0x7f74c019ecc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.338+0000 7f74c51f4640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74c01017a0 0x7f74c019c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.338+0000 7f74c51f4640 1 --2- 172.21.15.17:0/3720279979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c01005b0 0x7f74c006a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.339+0000 7f74c51f4640 1 -- 172.21.15.17:0/3720279979 >> 172.21.15.17:0/3720279979 conn(0x7f74c00fbb40 msgr2=0x7f74c0104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.339+0000 7f74c51f4640 1 -- 172.21.15.17:0/3720279979 shutdown_connections 2024-03-24T02:57:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.339+0000 7f74c51f4640 1 -- 172.21.15.17:0/3720279979 wait complete. 2024-03-24T02:57:57.350 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:57:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.452+0000 7f5a3e252640 1 -- 172.21.15.17:0/3126529883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a381016b0 msgr2=0x7f5a38101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.452+0000 7f5a3e252640 1 --2- 172.21.15.17:0/3126529883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a381016b0 0x7f5a38101ad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009900 tx=0x7f5a2802bc30 comp rx=0 tx=0).stop 2024-03-24T02:57:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.452+0000 7f5a3e252640 1 -- 172.21.15.17:0/3126529883 shutdown_connections 2024-03-24T02:57:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.452+0000 7f5a3e252640 1 --2- 172.21.15.17:0/3126529883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38103260 0x7f5a38109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 --2- 172.21.15.17:0/3126529883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a381028a0 0x7f5a38102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 --2- 172.21.15.17:0/3126529883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a381016b0 0x7f5a38101ad0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 -- 172.21.15.17:0/3126529883 >> 172.21.15.17:0/3126529883 conn(0x7f5a380fcc80 msgr2=0x7f5a380ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 -- 172.21.15.17:0/3126529883 shutdown_connections 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 -- 172.21.15.17:0/3126529883 wait complete. 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 Processor -- start 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 -- start start 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a381016b0 0x7f5a38075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a381028a0 0x7f5a38073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38103260 0x7f5a38073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3810d330 con 0x7f5a381016b0 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3810d1b0 con 0x7f5a381028a0 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a3e252640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3810d4b0 con 0x7f5a38103260 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a377fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a381016b0 0x7f5a38075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a377fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a381016b0 0x7f5a38075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51756/0 (socket says 172.21.15.17:51756) 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a377fe640 1 -- 172.21.15.17:0/3284255574 learned_addr learned my addr 172.21.15.17:0/3284255574 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a2ffff640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a381028a0 0x7f5a38073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.453+0000 7f5a37fff640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38103260 0x7f5a38073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.454+0000 7f5a377fe640 1 -- 172.21.15.17:0/3284255574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38103260 msgr2=0x7f5a38073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.454+0000 7f5a377fe640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38103260 0x7f5a38073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.454+0000 7f5a377fe640 1 -- 172.21.15.17:0/3284255574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a381028a0 msgr2=0x7f5a38073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.454+0000 7f5a377fe640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a381028a0 0x7f5a38073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.454+0000 7f5a377fe640 1 -- 172.21.15.17:0/3284255574 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a38074860 con 0x7f5a381016b0 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.454+0000 7f5a37fff640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38103260 0x7f5a38073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.454+0000 7f5a377fe640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a381016b0 0x7f5a38075450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005ab0 tx=0x7f5a28005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.454+0000 7f5a357fa640 1 -- 172.21.15.17:0/3284255574 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28041070 con 0x7f5a381016b0 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.454+0000 7f5a357fa640 1 -- 172.21.15.17:0/3284255574 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a280054e0 con 0x7f5a381016b0 2024-03-24T02:57:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.454+0000 7f5a3e252640 1 -- 172.21.15.17:0/3284255574 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a381a1a90 con 0x7f5a381016b0 2024-03-24T02:57:57.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.454+0000 7f5a357fa640 1 -- 172.21.15.17:0/3284255574 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28034440 con 0x7f5a381016b0 2024-03-24T02:57:57.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.454+0000 7f5a3e252640 1 -- 172.21.15.17:0/3284255574 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a381a2030 con 0x7f5a381016b0 2024-03-24T02:57:57.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.455+0000 7f5a3e252640 1 -- 172.21.15.17:0/3284255574 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a381089d0 con 0x7f5a381016b0 2024-03-24T02:57:57.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.457+0000 7f5a357fa640 1 -- 172.21.15.17:0/3284255574 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5a28044070 con 0x7f5a381016b0 2024-03-24T02:57:57.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.457+0000 7f5a357fa640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a240a2880 0x7f5a240a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:57.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.457+0000 7f5a2ffff640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a240a2880 0x7f5a240a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:57.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.457+0000 7f5a357fa640 1 -- 172.21.15.17:0/3284255574 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5a280f03e0 con 0x7f5a381016b0 2024-03-24T02:57:57.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.458+0000 7f5a2ffff640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a240a2880 0x7f5a240a4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5a380751c0 tx=0x7f5a2000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.461+0000 7f5a357fa640 1 -- 172.21.15.17:0/3284255574 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a280bd880 con 0x7f5a381016b0 2024-03-24T02:57:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.685+0000 7f5a3e252640 1 -- 172.21.15.17:0/3284255574 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a3802d070 con 0x7f5a381016b0 2024-03-24T02:57:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.686+0000 7f5a357fa640 1 -- 172.21.15.17:0/3284255574 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f5a280bd180 con 0x7f5a381016b0 2024-03-24T02:57:57.686 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:57:57.686 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-24T02:57:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 1 2024-03-24T02:57:57.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.687+0000 7f5a3e252640 1 -- 172.21.15.17:0/3284255574 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a240a2880 msgr2=0x7f5a240a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:57.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.687+0000 7f5a3e252640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a240a2880 0x7f5a240a4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5a380751c0 tx=0x7f5a2000b390 comp rx=0 tx=0).stop 2024-03-24T02:57:57.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.688+0000 7f5a3e252640 1 -- 172.21.15.17:0/3284255574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a381016b0 msgr2=0x7f5a38075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:57.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.688+0000 7f5a3e252640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a381016b0 0x7f5a38075450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005ab0 tx=0x7f5a28005dc0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.688+0000 7f5a3e252640 1 -- 172.21.15.17:0/3284255574 shutdown_connections 2024-03-24T02:57:57.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.688+0000 7f5a3e252640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a240a2880 0x7f5a240a4d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.688+0000 7f5a3e252640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38103260 0x7f5a38073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.688+0000 7f5a3e252640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a381028a0 0x7f5a38073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.688+0000 7f5a3e252640 1 --2- 172.21.15.17:0/3284255574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a381016b0 0x7f5a38075450 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.688+0000 7f5a3e252640 1 -- 172.21.15.17:0/3284255574 >> 172.21.15.17:0/3284255574 conn(0x7f5a380fcc80 msgr2=0x7f5a38105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.688+0000 7f5a3e252640 1 -- 172.21.15.17:0/3284255574 shutdown_connections 2024-03-24T02:57:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:57.688+0000 7f5a3e252640 1 -- 172.21.15.17:0/3284255574 wait complete. 2024-03-24T02:57:57.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:57 smithi188 ceph-mon[36206]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:57.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:57 smithi188 ceph-mon[36206]: Deploying daemon mds.l on smithi188 2024-03-24T02:57:57.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:57 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1582742436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:57:57.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:57 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3720279979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:57:57.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:57 smithi017 ceph-mon[31926]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:57.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:57 smithi017 ceph-mon[31926]: Deploying daemon mds.l on smithi188 2024-03-24T02:57:57.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:57 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1582742436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:57:57.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:57 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3720279979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:57:58.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:57 smithi019 ceph-mon[35944]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:58.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:57 smithi019 ceph-mon[35944]: Deploying daemon mds.l on smithi188 2024-03-24T02:57:58.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:57 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1582742436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:57:58.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:57 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3720279979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:57:58.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:58 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3284255574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:57:58.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:58 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3284255574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:57:59.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:58 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3284255574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:57:59.548 INFO:journalctl@ceph.mds.l.smithi188.stdout:Mar 24 02:57:59 smithi188 systemd[1]: Started Ceph mds.l for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:57:59.699 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:57:59.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 -- 172.21.15.17:0/1747089896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d681017b0 msgr2=0x7f1d68101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:59.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 --2- 172.21.15.17:0/1747089896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d681017b0 0x7f1d68101c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009900 tx=0x7f1d5c02bc30 comp rx=0 tx=0).stop 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 -- 172.21.15.17:0/1747089896 shutdown_connections 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 --2- 172.21.15.17:0/1747089896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d68102170 0x7f1d68108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 --2- 172.21.15.17:0/1747089896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d681017b0 0x7f1d68101c30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 --2- 172.21.15.17:0/1747089896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d681005c0 0x7f1d681009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 -- 172.21.15.17:0/1747089896 >> 172.21.15.17:0/1747089896 conn(0x7f1d680fbb30 msgr2=0x7f1d680fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 -- 172.21.15.17:0/1747089896 shutdown_connections 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 -- 172.21.15.17:0/1747089896 wait complete. 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 Processor -- start 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 -- start start 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d681005c0 0x7f1d68069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d681017b0 0x7f1d6819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.814+0000 7f1d6f84b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68102170 0x7f1d6819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:59.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6f84b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6810c240 con 0x7f1d68102170 2024-03-24T02:57:59.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6f84b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6810c0c0 con 0x7f1d681017b0 2024-03-24T02:57:59.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6f84b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6810c3c0 con 0x7f1d681005c0 2024-03-24T02:57:59.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6dde8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68102170 0x7f1d6819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:59.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6dde8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68102170 0x7f1d6819ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51768/0 (socket says 172.21.15.17:51768) 2024-03-24T02:57:59.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6dde8640 1 -- 172.21.15.17:0/2035595250 learned_addr learned my addr 172.21.15.17:0/2035595250 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:57:59.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6cde6640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d681017b0 0x7f1d6819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:59.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6d5e7640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d681005c0 0x7f1d68069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:59.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6dde8640 1 -- 172.21.15.17:0/2035595250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d681005c0 msgr2=0x7f1d68069370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:59.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6dde8640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d681005c0 0x7f1d68069370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:59.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6dde8640 1 -- 172.21.15.17:0/2035595250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d681017b0 msgr2=0x7f1d6819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:57:59.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6dde8640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d681017b0 0x7f1d6819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:57:59.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6dde8640 1 -- 172.21.15.17:0/2035595250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6819f630 con 0x7f1d68102170 2024-03-24T02:57:59.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6d5e7640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d681005c0 0x7f1d68069370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:57:59.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6dde8640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68102170 0x7f1d6819ed20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800eb10 tx=0x7f1d5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:59.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d667fc640 1 -- 172.21.15.17:0/2035595250 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58004b20 con 0x7f1d68102170 2024-03-24T02:57:59.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d667fc640 1 -- 172.21.15.17:0/2035595250 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d58004cc0 con 0x7f1d68102170 2024-03-24T02:57:59.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6f84b640 1 -- 172.21.15.17:0/2035595250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6819f8c0 con 0x7f1d68102170 2024-03-24T02:57:59.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d667fc640 1 -- 172.21.15.17:0/2035595250 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58010760 con 0x7f1d68102170 2024-03-24T02:57:59.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.815+0000 7f1d6f84b640 1 -- 172.21.15.17:0/2035595250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6819fd90 con 0x7f1d68102170 2024-03-24T02:57:59.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.816+0000 7f1d6f84b640 1 -- 172.21.15.17:0/2035595250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d681078e0 con 0x7f1d68102170 2024-03-24T02:57:59.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.817+0000 7f1d667fc640 1 -- 172.21.15.17:0/2035595250 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1d58005020 con 0x7f1d68102170 2024-03-24T02:57:59.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.818+0000 7f1d667fc640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1d500a2800 0x7f1d500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:57:59.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.818+0000 7f1d6d5e7640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1d500a2800 0x7f1d500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:57:59.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.818+0000 7f1d667fc640 1 -- 172.21.15.17:0/2035595250 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1d580cebe0 con 0x7f1d68102170 2024-03-24T02:57:59.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.819+0000 7f1d6d5e7640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1d500a2800 0x7f1d500a4cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1d68101610 tx=0x7f1d60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:57:59.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:57:59.823+0000 7f1d667fc640 1 -- 172.21.15.17:0/2035595250 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1d5809c120 con 0x7f1d68102170 2024-03-24T02:57:59.923 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:59 smithi019 ceph-mon[35944]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:59.923 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:59 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.188:6832/920982807,v1:172.21.15.188:6833/920982807] up:boot 2024-03-24T02:57:59.923 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:59 smithi019 ceph-mon[35944]: fsmap 1 up:standby 2024-03-24T02:57:59.923 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:59 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-24T02:57:59.923 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:59 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:59.923 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:59 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:59.923 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:59 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:59.923 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:59 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:57:59.923 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:59 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:57:59.924 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:57:59 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:57:59.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:59 smithi017 ceph-mon[31926]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:57:59.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:59 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.188:6832/920982807,v1:172.21.15.188:6833/920982807] up:boot 2024-03-24T02:57:59.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:59 smithi017 ceph-mon[31926]: fsmap 1 up:standby 2024-03-24T02:57:59.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-24T02:57:59.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:59.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:59.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:57:59.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:57:59.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:57:59.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:57:59 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:00.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.046+0000 7f1d6f84b640 1 -- 172.21.15.17:0/2035595250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d6802d250 con 0x7f1d68102170 2024-03-24T02:58:00.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.046+0000 7f1d667fc640 1 -- 172.21.15.17:0/2035595250 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f1d5809ba20 con 0x7f1d68102170 2024-03-24T02:58:00.047 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:00.047 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-24T02:58:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 2 2024-03-24T02:58:00.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:59 smithi188 ceph-mon[36206]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:00.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:59 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.188:6832/920982807,v1:172.21.15.188:6833/920982807] up:boot 2024-03-24T02:58:00.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:59 smithi188 ceph-mon[36206]: fsmap 1 up:standby 2024-03-24T02:58:00.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:59 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-24T02:58:00.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:59 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:00.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:59 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:00.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:59 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:00.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:59 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:00.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:59 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:00.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:57:59 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.048+0000 7f1d6f84b640 1 -- 172.21.15.17:0/2035595250 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1d500a2800 msgr2=0x7f1d500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.048+0000 7f1d6f84b640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1d500a2800 0x7f1d500a4cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1d68101610 tx=0x7f1d60005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.048+0000 7f1d6f84b640 1 -- 172.21.15.17:0/2035595250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68102170 msgr2=0x7f1d6819ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.048+0000 7f1d6f84b640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68102170 0x7f1d6819ed20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800eb10 tx=0x7f1d5800c2d0 comp rx=0 tx=0).stop 2024-03-24T02:58:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.048+0000 7f1d6f84b640 1 -- 172.21.15.17:0/2035595250 shutdown_connections 2024-03-24T02:58:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.048+0000 7f1d6f84b640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1d500a2800 0x7f1d500a4cc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.048+0000 7f1d6f84b640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68102170 0x7f1d6819ed20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.048+0000 7f1d6f84b640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d681017b0 0x7f1d6819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.048+0000 7f1d6f84b640 1 --2- 172.21.15.17:0/2035595250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d681005c0 0x7f1d68069370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.048+0000 7f1d6f84b640 1 -- 172.21.15.17:0/2035595250 >> 172.21.15.17:0/2035595250 conn(0x7f1d680fbb30 msgr2=0x7f1d68104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.048+0000 7f1d6f84b640 1 -- 172.21.15.17:0/2035595250 shutdown_connections 2024-03-24T02:58:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:00.048+0000 7f1d6f84b640 1 -- 172.21.15.17:0/2035595250 wait complete. 2024-03-24T02:58:00.953 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:00 smithi019 ceph-mon[35944]: Deploying daemon mds.k on smithi019 2024-03-24T02:58:00.953 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:00 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2035595250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:00.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:00 smithi017 ceph-mon[31926]: Deploying daemon mds.k on smithi019 2024-03-24T02:58:00.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:00 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2035595250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:01.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:00 smithi188 ceph-mon[36206]: Deploying daemon mds.k on smithi019 2024-03-24T02:58:01.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:00 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2035595250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:01.901 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:01 smithi019 ceph-mon[35944]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:01.901 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 02:58:01 smithi019 podman[65532]: 2024-03-24 02:58:01.777259968 +0000 UTC m=+0.018122045 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T02:58:01.902 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 02:58:01 smithi019 podman[65532]: 2024-03-24 02:58:01.902296729 +0000 UTC m=+0.143158803 container create 535b6a63956fdf12270ae3946bb95a03ac362f58f54ea2f70a9081653fc18091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-24T02:58:01.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:01 smithi017 ceph-mon[31926]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:02.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:01 smithi188 ceph-mon[36206]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:02.060 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:02.163 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 02:58:02 smithi019 podman[65532]: 2024-03-24 02:58:02.138494015 +0000 UTC m=+0.379356096 container init 535b6a63956fdf12270ae3946bb95a03ac362f58f54ea2f70a9081653fc18091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-24T02:58:02.209 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 02:58:02 smithi019 podman[65532]: 2024-03-24 02:58:02.141210543 +0000 UTC m=+0.382072618 container start 535b6a63956fdf12270ae3946bb95a03ac362f58f54ea2f70a9081653fc18091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T02:58:02.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.176+0000 7f955abc4640 1 -- 172.21.15.17:0/2761738947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9554102170 msgr2=0x7f9554108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:02.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.176+0000 7f955abc4640 1 --2- 172.21.15.17:0/2761738947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9554102170 0x7f9554108a00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f954c009ca0 tx=0x7f954c02c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:02.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.176+0000 7f955abc4640 1 -- 172.21.15.17:0/2761738947 shutdown_connections 2024-03-24T02:58:02.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.176+0000 7f955abc4640 1 --2- 172.21.15.17:0/2761738947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9554102170 0x7f9554108a00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:02.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.176+0000 7f955abc4640 1 --2- 172.21.15.17:0/2761738947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541017b0 0x7f9554101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:02.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.176+0000 7f955abc4640 1 --2- 172.21.15.17:0/2761738947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95541005c0 0x7f95541009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:02.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.176+0000 7f955abc4640 1 -- 172.21.15.17:0/2761738947 >> 172.21.15.17:0/2761738947 conn(0x7f95540fbb30 msgr2=0x7f95540fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:02.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.176+0000 7f955abc4640 1 -- 172.21.15.17:0/2761738947 shutdown_connections 2024-03-24T02:58:02.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.176+0000 7f955abc4640 1 -- 172.21.15.17:0/2761738947 wait complete. 2024-03-24T02:58:02.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f955abc4640 1 Processor -- start 2024-03-24T02:58:02.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f955abc4640 1 -- start start 2024-03-24T02:58:02.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f955abc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541005c0 0x7f9554069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:02.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f955abc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95541017b0 0x7f955419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:02.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f955abc4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9554102170 0x7f955419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:02.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f955abc4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955410c220 con 0x7f9554102170 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f955abc4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955410c0a0 con 0x7f95541005c0 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f955abc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955410c3a0 con 0x7f95541017b0 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f9559161640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9554102170 0x7f955419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f9559161640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9554102170 0x7f955419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57158/0 (socket says 172.21.15.17:57158) 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f9559161640 1 -- 172.21.15.17:0/3762659379 learned_addr learned my addr 172.21.15.17:0/3762659379 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f9553fff640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95541017b0 0x7f955419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f9558960640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541005c0 0x7f9554069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f9559161640 1 -- 172.21.15.17:0/3762659379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95541017b0 msgr2=0x7f955419c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f9559161640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95541017b0 0x7f955419c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f9559161640 1 -- 172.21.15.17:0/3762659379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541005c0 msgr2=0x7f9554069370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f9559161640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541005c0 0x7f9554069370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f9559161640 1 -- 172.21.15.17:0/3762659379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955419f5e0 con 0x7f9554102170 2024-03-24T02:58:02.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.177+0000 7f9558960640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541005c0 0x7f9554069370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:02.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.178+0000 7f9559161640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9554102170 0x7f955419ecd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f954c037040 tx=0x7f954c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:02.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.178+0000 7f9551ffb640 1 -- 172.21.15.17:0/3762659379 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c041020 con 0x7f9554102170 2024-03-24T02:58:02.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.178+0000 7f9551ffb640 1 -- 172.21.15.17:0/3762659379 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f954c034d90 con 0x7f9554102170 2024-03-24T02:58:02.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.178+0000 7f955abc4640 1 -- 172.21.15.17:0/3762659379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955419f810 con 0x7f9554102170 2024-03-24T02:58:02.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.178+0000 7f9551ffb640 1 -- 172.21.15.17:0/3762659379 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c039040 con 0x7f9554102170 2024-03-24T02:58:02.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.178+0000 7f955abc4640 1 -- 172.21.15.17:0/3762659379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955419fd50 con 0x7f9554102170 2024-03-24T02:58:02.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.179+0000 7f955abc4640 1 -- 172.21.15.17:0/3762659379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95541078e0 con 0x7f9554102170 2024-03-24T02:58:02.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.181+0000 7f9551ffb640 1 -- 172.21.15.17:0/3762659379 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f954c044020 con 0x7f9554102170 2024-03-24T02:58:02.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.182+0000 7f9551ffb640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f95440a2880 0x7f95440a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:02.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.182+0000 7f9558960640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f95440a2880 0x7f95440a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:02.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.182+0000 7f9551ffb640 1 -- 172.21.15.17:0/3762659379 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f954c0f08d0 con 0x7f9554102170 2024-03-24T02:58:02.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.182+0000 7f9558960640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f95440a2880 0x7f95440a4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9548005d00 tx=0x7f9548005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:02.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.185+0000 7f9551ffb640 1 -- 172.21.15.17:0/3762659379 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f954c04b020 con 0x7f9554102170 2024-03-24T02:58:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.413+0000 7f955abc4640 1 -- 172.21.15.17:0/3762659379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f955402d230 con 0x7f9554102170 2024-03-24T02:58:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.413+0000 7f9551ffb640 1 -- 172.21.15.17:0/3762659379 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f954c0bde10 con 0x7f9554102170 2024-03-24T02:58:02.414 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:02.414 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-24T02:58:02.415 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 2 2024-03-24T02:58:02.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.416+0000 7f955abc4640 1 -- 172.21.15.17:0/3762659379 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f95440a2880 msgr2=0x7f95440a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:02.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.416+0000 7f955abc4640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f95440a2880 0x7f95440a4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9548005d00 tx=0x7f9548005c90 comp rx=0 tx=0).stop 2024-03-24T02:58:02.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.416+0000 7f955abc4640 1 -- 172.21.15.17:0/3762659379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9554102170 msgr2=0x7f955419ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:02.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.416+0000 7f955abc4640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9554102170 0x7f955419ecd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f954c037040 tx=0x7f954c032f20 comp rx=0 tx=0).stop 2024-03-24T02:58:02.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.416+0000 7f955abc4640 1 -- 172.21.15.17:0/3762659379 shutdown_connections 2024-03-24T02:58:02.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.416+0000 7f955abc4640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f95440a2880 0x7f95440a4d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:02.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.416+0000 7f955abc4640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9554102170 0x7f955419ecd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:02.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.416+0000 7f955abc4640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95541017b0 0x7f955419c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:02.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.416+0000 7f955abc4640 1 --2- 172.21.15.17:0/3762659379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541005c0 0x7f9554069370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:02.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.416+0000 7f955abc4640 1 -- 172.21.15.17:0/3762659379 >> 172.21.15.17:0/3762659379 conn(0x7f95540fbb30 msgr2=0x7f9554104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:02.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.416+0000 7f955abc4640 1 -- 172.21.15.17:0/3762659379 shutdown_connections 2024-03-24T02:58:02.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:02.416+0000 7f955abc4640 1 -- 172.21.15.17:0/3762659379 wait complete. 2024-03-24T02:58:02.570 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 02:58:02 smithi019 ceph-mds[65552]: -- 172.21.15.19:0/3035492482 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c2f5a1a40 con 0x555c2f574c00 2024-03-24T02:58:02.570 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 02:58:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k[65548]: starting mds.k at 2024-03-24T02:58:02.570 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 02:58:02 smithi019 bash[65532]: 535b6a63956fdf12270ae3946bb95a03ac362f58f54ea2f70a9081653fc18091 2024-03-24T02:58:02.570 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 02:58:02 smithi019 systemd[1]: Started Ceph mds.k for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:58:03.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:03 smithi188 ceph-mon[36206]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:03 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:03 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:03 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:03 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:03 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:03 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:03 smithi188 ceph-mon[36206]: Deploying daemon mds.j on smithi017 2024-03-24T02:58:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:03 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3762659379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:03 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.19:6832/3469034636,v1:172.21.15.19:6833/3469034636] up:boot 2024-03-24T02:58:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:03 smithi188 ceph-mon[36206]: fsmap 2 up:standby 2024-03-24T02:58:03.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:03 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-24T02:58:03.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:03 smithi019 ceph-mon[35944]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:03.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:03 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:03.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:03 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:03.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:03 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:03.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:03 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:03.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:03 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:03.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:03 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:03.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:03 smithi019 ceph-mon[35944]: Deploying daemon mds.j on smithi017 2024-03-24T02:58:03.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:03 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3762659379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:03.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:03 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.19:6832/3469034636,v1:172.21.15.19:6833/3469034636] up:boot 2024-03-24T02:58:03.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:03 smithi019 ceph-mon[35944]: fsmap 2 up:standby 2024-03-24T02:58:03.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:03 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-24T02:58:03.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:03 smithi017 ceph-mon[31926]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:03.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:03.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:03.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:03.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:03.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:03.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:03.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:03 smithi017 ceph-mon[31926]: Deploying daemon mds.j on smithi017 2024-03-24T02:58:03.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:03 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3762659379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:03.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:03 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.19:6832/3469034636,v1:172.21.15.19:6833/3469034636] up:boot 2024-03-24T02:58:03.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:03 smithi017 ceph-mon[31926]: fsmap 2 up:standby 2024-03-24T02:58:03.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:03 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-24T02:58:04.428 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:04.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.625+0000 7f847224b640 1 -- 172.21.15.17:0/1957588602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c103290 msgr2=0x7f846c109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.625+0000 7f847224b640 1 --2- 172.21.15.17:0/1957588602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c103290 0x7f846c109b20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8464009ca0 tx=0x7f846402c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.625+0000 7f847224b640 1 -- 172.21.15.17:0/1957588602 shutdown_connections 2024-03-24T02:58:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.625+0000 7f847224b640 1 --2- 172.21.15.17:0/1957588602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c103290 0x7f846c109b20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.625+0000 7f847224b640 1 --2- 172.21.15.17:0/1957588602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f846c1028d0 0x7f846c102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.625+0000 7f847224b640 1 --2- 172.21.15.17:0/1957588602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f846c1016e0 0x7f846c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.625+0000 7f847224b640 1 -- 172.21.15.17:0/1957588602 >> 172.21.15.17:0/1957588602 conn(0x7f846c0fcc50 msgr2=0x7f846c0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.625+0000 7f847224b640 1 -- 172.21.15.17:0/1957588602 shutdown_connections 2024-03-24T02:58:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.625+0000 7f847224b640 1 -- 172.21.15.17:0/1957588602 wait complete. 2024-03-24T02:58:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.625+0000 7f847224b640 1 Processor -- start 2024-03-24T02:58:04.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f847224b640 1 -- start start 2024-03-24T02:58:04.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f847224b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f846c1016e0 0x7f846c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:04.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f847224b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c1028d0 0x7f846c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:04.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f847224b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f846c103290 0x7f846c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:04.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f847224b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c10d3e0 con 0x7f846c1028d0 2024-03-24T02:58:04.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f847224b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c10d260 con 0x7f846c103290 2024-03-24T02:58:04.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f847224b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c10d560 con 0x7f846c1016e0 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f846affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c1028d0 0x7f846c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f846affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c1028d0 0x7f846c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57180/0 (socket says 172.21.15.17:57180) 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f846affd640 1 -- 172.21.15.17:0/11714179 learned_addr learned my addr 172.21.15.17:0/11714179 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f846bfff640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f846c103290 0x7f846c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f846b7fe640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f846c1016e0 0x7f846c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f846affd640 1 -- 172.21.15.17:0/11714179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f846c1016e0 msgr2=0x7f846c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f846affd640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f846c1016e0 0x7f846c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f846affd640 1 -- 172.21.15.17:0/11714179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f846c103290 msgr2=0x7f846c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f846affd640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f846c103290 0x7f846c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f846affd640 1 -- 172.21.15.17:0/11714179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c19f640 con 0x7f846c1028d0 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.626+0000 7f846bfff640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f846c103290 0x7f846c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.627+0000 7f846affd640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c1028d0 0x7f846c197f60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f846000b4a0 tx=0x7f846000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.627+0000 7f8468ff9640 1 -- 172.21.15.17:0/11714179 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460014070 con 0x7f846c1028d0 2024-03-24T02:58:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.627+0000 7f847224b640 1 -- 172.21.15.17:0/11714179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846c19f930 con 0x7f846c1028d0 2024-03-24T02:58:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.627+0000 7f8468ff9640 1 -- 172.21.15.17:0/11714179 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8460010af0 con 0x7f846c1028d0 2024-03-24T02:58:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.627+0000 7f8468ff9640 1 -- 172.21.15.17:0/11714179 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460017d10 con 0x7f846c1028d0 2024-03-24T02:58:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.627+0000 7f847224b640 1 -- 172.21.15.17:0/11714179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846c19fed0 con 0x7f846c1028d0 2024-03-24T02:58:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.628+0000 7f847224b640 1 -- 172.21.15.17:0/11714179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846c108a00 con 0x7f846c1028d0 2024-03-24T02:58:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.630+0000 7f8468ff9640 1 -- 172.21.15.17:0/11714179 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8460017070 con 0x7f846c1028d0 2024-03-24T02:58:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.630+0000 7f8468ff9640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f84540a2800 0x7f84540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.630+0000 7f8468ff9640 1 -- 172.21.15.17:0/11714179 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f84600cd840 con 0x7f846c1028d0 2024-03-24T02:58:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.630+0000 7f846b7fe640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f84540a2800 0x7f84540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.631+0000 7f846b7fe640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f84540a2800 0x7f84540a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f846c102730 tx=0x7f845c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:04.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.636+0000 7f8468ff9640 1 -- 172.21.15.17:0/11714179 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f846009a730 con 0x7f846c1028d0 2024-03-24T02:58:04.706 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 02:58:04 smithi017 systemd[1]: Starting Ceph mds.j for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:58:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.864+0000 7f847224b640 1 -- 172.21.15.17:0/11714179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f846c02d200 con 0x7f846c1028d0 2024-03-24T02:58:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.865+0000 7f8468ff9640 1 -- 172.21.15.17:0/11714179 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2116 (secure 0 0 0) 0x7f846009e7d0 con 0x7f846c1028d0 2024-03-24T02:58:04.865 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:04.865 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-24T02:58:04.866 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 3 2024-03-24T02:58:04.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.867+0000 7f847224b640 1 -- 172.21.15.17:0/11714179 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f84540a2800 msgr2=0x7f84540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.867+0000 7f847224b640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f84540a2800 0x7f84540a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f846c102730 tx=0x7f845c005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.867+0000 7f847224b640 1 -- 172.21.15.17:0/11714179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c1028d0 msgr2=0x7f846c197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.867+0000 7f847224b640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c1028d0 0x7f846c197f60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f846000b4a0 tx=0x7f846000b960 comp rx=0 tx=0).stop 2024-03-24T02:58:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.867+0000 7f847224b640 1 -- 172.21.15.17:0/11714179 shutdown_connections 2024-03-24T02:58:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.867+0000 7f847224b640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f84540a2800 0x7f84540a4cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.867+0000 7f847224b640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f846c103290 0x7f846c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.867+0000 7f847224b640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c1028d0 0x7f846c197f60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.867+0000 7f847224b640 1 --2- 172.21.15.17:0/11714179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f846c1016e0 0x7f846c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.867+0000 7f847224b640 1 -- 172.21.15.17:0/11714179 >> 172.21.15.17:0/11714179 conn(0x7f846c0fcc50 msgr2=0x7f846c105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.867+0000 7f847224b640 1 -- 172.21.15.17:0/11714179 shutdown_connections 2024-03-24T02:58:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:04.867+0000 7f847224b640 1 -- 172.21.15.17:0/11714179 wait complete. 2024-03-24T02:58:05.051 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 02:58:04 smithi017 podman[72576]: 2024-03-24 02:58:04.683024031 +0000 UTC m=+0.017765577 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T02:58:05.051 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 02:58:04 smithi017 podman[72576]: 2024-03-24 02:58:04.80806941 +0000 UTC m=+0.142810940 container create 37509107601a2dc6fc2c58c43304467276652060dae8f203bef6cdb45b780904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T02:58:05.303 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 02:58:05 smithi017 podman[72576]: 2024-03-24 02:58:05.052806093 +0000 UTC m=+0.387547634 container init 37509107601a2dc6fc2c58c43304467276652060dae8f203bef6cdb45b780904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-24T02:58:05.303 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 02:58:05 smithi017 podman[72576]: 2024-03-24 02:58:05.055950455 +0000 UTC m=+0.390691995 container start 37509107601a2dc6fc2c58c43304467276652060dae8f203bef6cdb45b780904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-24T02:58:05.303 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 02:58:05 smithi017 ceph-mds[72597]: -- 172.21.15.17:0/486531104 <== mon.0 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b262503a40 con 0x55b2624d6800 2024-03-24T02:58:05.303 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 02:58:05 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-j[72593]: starting mds.j at 2024-03-24T02:58:05.303 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 02:58:05 smithi017 bash[72576]: 37509107601a2dc6fc2c58c43304467276652060dae8f203bef6cdb45b780904 2024-03-24T02:58:05.303 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 02:58:05 smithi017 systemd[1]: Started Ceph mds.j for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:58:05.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:05 smithi019 ceph-mon[35944]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:05.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:05 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/11714179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:05.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:05.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:05.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:05.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:05.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:05.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:05 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:05 smithi017 ceph-mon[31926]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:05 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/11714179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:05 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:05.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:05 smithi188 ceph-mon[36206]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:05.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:05 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/11714179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:05.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:05.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:05.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:05.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:05.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:05.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:05 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:06.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:06 smithi017 ceph-mon[31926]: Deploying daemon mds.d on smithi017 2024-03-24T02:58:06.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:06 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.17:6834/2854025639,v1:172.21.15.17:6835/2854025639] up:boot 2024-03-24T02:58:06.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:06 smithi017 ceph-mon[31926]: fsmap 3 up:standby 2024-03-24T02:58:06.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:06 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-24T02:58:06.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:06 smithi017 ceph-mon[31926]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:06.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:06 smithi019 ceph-mon[35944]: Deploying daemon mds.d on smithi017 2024-03-24T02:58:06.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:06 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.17:6834/2854025639,v1:172.21.15.17:6835/2854025639] up:boot 2024-03-24T02:58:06.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:06 smithi019 ceph-mon[35944]: fsmap 3 up:standby 2024-03-24T02:58:06.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:06 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-24T02:58:06.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:06 smithi019 ceph-mon[35944]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:06.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:06 smithi188 ceph-mon[36206]: Deploying daemon mds.d on smithi017 2024-03-24T02:58:06.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:06 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.17:6834/2854025639,v1:172.21.15.17:6835/2854025639] up:boot 2024-03-24T02:58:06.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:06 smithi188 ceph-mon[36206]: fsmap 3 up:standby 2024-03-24T02:58:06.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:06 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-24T02:58:06.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:06 smithi188 ceph-mon[36206]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:06.879 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.995+0000 7fd705390640 1 -- 172.21.15.17:0/2288778065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001017b0 msgr2=0x7fd700101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.995+0000 7fd705390640 1 --2- 172.21.15.17:0/2288778065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001017b0 0x7fd700101c30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009900 tx=0x7fd6e802bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.995+0000 7fd705390640 1 -- 172.21.15.17:0/2288778065 shutdown_connections 2024-03-24T02:58:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.995+0000 7fd705390640 1 --2- 172.21.15.17:0/2288778065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd700102170 0x7fd700108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.995+0000 7fd705390640 1 --2- 172.21.15.17:0/2288778065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7001017b0 0x7fd700101c30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.995+0000 7fd705390640 1 --2- 172.21.15.17:0/2288778065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7001005c0 0x7fd7001009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.995+0000 7fd705390640 1 -- 172.21.15.17:0/2288778065 >> 172.21.15.17:0/2288778065 conn(0x7fd7000fbb30 msgr2=0x7fd7000fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.995+0000 7fd705390640 1 -- 172.21.15.17:0/2288778065 shutdown_connections 2024-03-24T02:58:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.995+0000 7fd705390640 1 -- 172.21.15.17:0/2288778065 wait complete. 2024-03-24T02:58:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd705390640 1 Processor -- start 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd705390640 1 -- start start 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd705390640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7001005c0 0x7fd700069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd705390640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7001017b0 0x7fd70019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd705390640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd700102170 0x7fd70019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd705390640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70010c2a0 con 0x7fd700102170 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd705390640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70010c120 con 0x7fd7001017b0 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd705390640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70010c420 con 0x7fd7001005c0 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd6ff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd700102170 0x7fd70019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd6ff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd700102170 0x7fd70019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57222/0 (socket says 172.21.15.17:57222) 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd6ff7fe640 1 -- 172.21.15.17:0/4135175526 learned_addr learned my addr 172.21.15.17:0/4135175526 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd6f7fff640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7001017b0 0x7fd70019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.996+0000 7fd6feffd640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7001005c0 0x7fd700069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.997+0000 7fd6ff7fe640 1 -- 172.21.15.17:0/4135175526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7001005c0 msgr2=0x7fd700069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.997+0000 7fd6ff7fe640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7001005c0 0x7fd700069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.997+0000 7fd6ff7fe640 1 -- 172.21.15.17:0/4135175526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7001017b0 msgr2=0x7fd70019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.997+0000 7fd6ff7fe640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7001017b0 0x7fd70019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.997+0000 7fd6ff7fe640 1 -- 172.21.15.17:0/4135175526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70019f5e0 con 0x7fd700102170 2024-03-24T02:58:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.997+0000 7fd6ff7fe640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd700102170 0x7fd70019ed20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800b970 tx=0x7fd6f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.997+0000 7fd6fcff9640 1 -- 172.21.15.17:0/4135175526 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8019070 con 0x7fd700102170 2024-03-24T02:58:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.997+0000 7fd6fcff9640 1 -- 172.21.15.17:0/4135175526 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6f800cb40 con 0x7fd700102170 2024-03-24T02:58:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.997+0000 7fd705390640 1 -- 172.21.15.17:0/4135175526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70019f8d0 con 0x7fd700102170 2024-03-24T02:58:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.997+0000 7fd6fcff9640 1 -- 172.21.15.17:0/4135175526 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8007650 con 0x7fd700102170 2024-03-24T02:58:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.997+0000 7fd705390640 1 -- 172.21.15.17:0/4135175526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70019fdf0 con 0x7fd700102170 2024-03-24T02:58:07.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.998+0000 7fd705390640 1 -- 172.21.15.17:0/4135175526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd700107880 con 0x7fd700102170 2024-03-24T02:58:07.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:06.999+0000 7fd6fcff9640 1 -- 172.21.15.17:0/4135175526 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd6f801c070 con 0x7fd700102170 2024-03-24T02:58:07.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.000+0000 7fd6fcff9640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd6ec0a2800 0x7fd6ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:07.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.000+0000 7fd6feffd640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd6ec0a2800 0x7fd6ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:07.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.000+0000 7fd6fcff9640 1 -- 172.21.15.17:0/4135175526 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd6f80cdfb0 con 0x7fd700102170 2024-03-24T02:58:07.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.000+0000 7fd6feffd640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd6ec0a2800 0x7fd6ec0a4cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd700101610 tx=0x7fd6f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:07.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.004+0000 7fd6fcff9640 1 -- 172.21.15.17:0/4135175526 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd6f809aea0 con 0x7fd700102170 2024-03-24T02:58:07.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.241+0000 7fd705390640 1 -- 172.21.15.17:0/4135175526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd70002d070 con 0x7fd700102170 2024-03-24T02:58:07.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.242+0000 7fd6fcff9640 1 -- 172.21.15.17:0/4135175526 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2907 (secure 0 0 0) 0x7fd6f809ef40 con 0x7fd700102170 2024-03-24T02:58:07.242 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:07.243 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-24T02:58:07.243 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 4 2024-03-24T02:58:07.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.245+0000 7fd705390640 1 -- 172.21.15.17:0/4135175526 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd6ec0a2800 msgr2=0x7fd6ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:07.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.245+0000 7fd705390640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd6ec0a2800 0x7fd6ec0a4cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd700101610 tx=0x7fd6f0005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:07.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.245+0000 7fd705390640 1 -- 172.21.15.17:0/4135175526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd700102170 msgr2=0x7fd70019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:07.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.245+0000 7fd705390640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd700102170 0x7fd70019ed20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800b970 tx=0x7fd6f800be30 comp rx=0 tx=0).stop 2024-03-24T02:58:07.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.245+0000 7fd705390640 1 -- 172.21.15.17:0/4135175526 shutdown_connections 2024-03-24T02:58:07.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.245+0000 7fd705390640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd6ec0a2800 0x7fd6ec0a4cc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:07.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.245+0000 7fd705390640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd700102170 0x7fd70019ed20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:07.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.245+0000 7fd705390640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7001017b0 0x7fd70019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:07.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.245+0000 7fd705390640 1 --2- 172.21.15.17:0/4135175526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7001005c0 0x7fd700069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:07.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.245+0000 7fd705390640 1 -- 172.21.15.17:0/4135175526 >> 172.21.15.17:0/4135175526 conn(0x7fd7000fbb30 msgr2=0x7fd700104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:07.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.245+0000 7fd705390640 1 -- 172.21.15.17:0/4135175526 shutdown_connections 2024-03-24T02:58:07.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:07.245+0000 7fd705390640 1 -- 172.21.15.17:0/4135175526 wait complete. 2024-03-24T02:58:07.523 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:07 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4135175526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:07.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:07 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4135175526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:07.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:07 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4135175526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:08.315 INFO:journalctl@ceph.mds.d.smithi017.stdout:Mar 24 02:58:08 smithi017 systemd[1]: Started Ceph mds.d for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:58:08.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:08 smithi019 ceph-mon[35944]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:08.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:08.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:08.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:08.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:08.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:08.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:08.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:08 smithi017 ceph-mon[31926]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:08.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:08.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:08.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:08.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:08.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:08.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:08.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:08 smithi188 ceph-mon[36206]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:09.257 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:09.317 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:09 smithi019 ceph-mon[35944]: Deploying daemon mds.e on smithi019 2024-03-24T02:58:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.379+0000 7fa5aeaea640 1 -- 172.21.15.17:0/4245934795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a81017d0 msgr2=0x7fa5a8101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.379+0000 7fa5aeaea640 1 --2- 172.21.15.17:0/4245934795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a81017d0 0x7fa5a8101c50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa594009900 tx=0x7fa59402bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.379+0000 7fa5aeaea640 1 -- 172.21.15.17:0/4245934795 shutdown_connections 2024-03-24T02:58:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.379+0000 7fa5aeaea640 1 --2- 172.21.15.17:0/4245934795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a8102190 0x7fa5a8108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.379+0000 7fa5aeaea640 1 --2- 172.21.15.17:0/4245934795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a81017d0 0x7fa5a8101c50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.379+0000 7fa5aeaea640 1 --2- 172.21.15.17:0/4245934795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a81005e0 0x7fa5a8100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.379+0000 7fa5aeaea640 1 -- 172.21.15.17:0/4245934795 >> 172.21.15.17:0/4245934795 conn(0x7fa5a80fbb50 msgr2=0x7fa5a80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.379+0000 7fa5aeaea640 1 -- 172.21.15.17:0/4245934795 shutdown_connections 2024-03-24T02:58:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.379+0000 7fa5aeaea640 1 -- 172.21.15.17:0/4245934795 wait complete. 2024-03-24T02:58:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5aeaea640 1 Processor -- start 2024-03-24T02:58:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5aeaea640 1 -- start start 2024-03-24T02:58:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5aeaea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a81005e0 0x7fa5a8104570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:09.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5aeaea640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a81017d0 0x7fa5a8102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:09.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5aeaea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a8102190 0x7fa5a81030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:09.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5aeaea640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a810c260 con 0x7fa5a81017d0 2024-03-24T02:58:09.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5aeaea640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a810c0e0 con 0x7fa5a81005e0 2024-03-24T02:58:09.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5aeaea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a810c3e0 con 0x7fa5a8102190 2024-03-24T02:58:09.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5a7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a81017d0 0x7fa5a8102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:09.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5ac886640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a81005e0 0x7fa5a8104570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:09.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5a7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a81017d0 0x7fa5a8102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57266/0 (socket says 172.21.15.17:57266) 2024-03-24T02:58:09.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5ac886640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a81005e0 0x7fa5a8104570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:58430/0 (socket says 172.21.15.17:58430) 2024-03-24T02:58:09.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5a7fff640 1 -- 172.21.15.17:0/1562967577 learned_addr learned my addr 172.21.15.17:0/1562967577 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:09.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5ad087640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a8102190 0x7fa5a81030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:09.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5ac886640 1 -- 172.21.15.17:0/1562967577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a8102190 msgr2=0x7fa5a81030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:09.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5ac886640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a8102190 0x7fa5a81030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:09.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5ac886640 1 -- 172.21.15.17:0/1562967577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a81017d0 msgr2=0x7fa5a8102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:09.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5ac886640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a81017d0 0x7fa5a8102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:09.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5ac886640 1 -- 172.21.15.17:0/1562967577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a8103980 con 0x7fa5a81005e0 2024-03-24T02:58:09.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5a7fff640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a81017d0 0x7fa5a8102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:58:09.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.380+0000 7fa5ac886640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a81005e0 0x7fa5a8104570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00e9c0 tx=0x7fa59c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:09.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.381+0000 7fa5a5ffb640 1 -- 172.21.15.17:0/1562967577 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c00ccc0 con 0x7fa5a81005e0 2024-03-24T02:58:09.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.381+0000 7fa5aeaea640 1 -- 172.21.15.17:0/1562967577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a81a1b50 con 0x7fa5a81005e0 2024-03-24T02:58:09.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.381+0000 7fa5a5ffb640 1 -- 172.21.15.17:0/1562967577 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa59c0035f0 con 0x7fa5a81005e0 2024-03-24T02:58:09.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.381+0000 7fa5a5ffb640 1 -- 172.21.15.17:0/1562967577 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c00ce60 con 0x7fa5a81005e0 2024-03-24T02:58:09.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.381+0000 7fa5aeaea640 1 -- 172.21.15.17:0/1562967577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a81a20f0 con 0x7fa5a81005e0 2024-03-24T02:58:09.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.382+0000 7fa5aeaea640 1 -- 172.21.15.17:0/1562967577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a81a22c0 con 0x7fa5a81005e0 2024-03-24T02:58:09.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.383+0000 7fa5a5ffb640 1 -- 172.21.15.17:0/1562967577 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa59c0037a0 con 0x7fa5a81005e0 2024-03-24T02:58:09.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.383+0000 7fa5a5ffb640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa5980a2800 0x7fa5980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:09.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.384+0000 7fa5a5ffb640 1 -- 172.21.15.17:0/1562967577 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa59c0cea30 con 0x7fa5a81005e0 2024-03-24T02:58:09.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.384+0000 7fa5a7fff640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa5980a2800 0x7fa5980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:09.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.384+0000 7fa5a7fff640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa5980a2800 0x7fa5980a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa5a81042e0 tx=0x7fa59402c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:09.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.388+0000 7fa5a5ffb640 1 -- 172.21.15.17:0/1562967577 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa59c09bed0 con 0x7fa5a81005e0 2024-03-24T02:58:09.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:09 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.17:6836/3530669946,v1:172.21.15.17:6837/3530669946] up:boot 2024-03-24T02:58:09.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:09 smithi019 ceph-mon[35944]: fsmap 4 up:standby 2024-03-24T02:58:09.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:09 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-24T02:58:09.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.610+0000 7fa5aeaea640 1 -- 172.21.15.17:0/1562967577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5a81a2490 con 0x7fa5a81005e0 2024-03-24T02:58:09.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.610+0000 7fa5a5ffb640 1 -- 172.21.15.17:0/1562967577 <== mon.2 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3698 (secure 0 0 0) 0x7fa59c09b7d0 con 0x7fa5a81005e0 2024-03-24T02:58:09.611 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:09.611 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-24T02:58:09.611 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 5 2024-03-24T02:58:09.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.613+0000 7fa5aeaea640 1 -- 172.21.15.17:0/1562967577 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa5980a2800 msgr2=0x7fa5980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:09.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.613+0000 7fa5aeaea640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa5980a2800 0x7fa5980a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa5a81042e0 tx=0x7fa59402c880 comp rx=0 tx=0).stop 2024-03-24T02:58:09.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.613+0000 7fa5aeaea640 1 -- 172.21.15.17:0/1562967577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a81005e0 msgr2=0x7fa5a8104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:09.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.613+0000 7fa5aeaea640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a81005e0 0x7fa5a8104570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00e9c0 tx=0x7fa59c00ee80 comp rx=0 tx=0).stop 2024-03-24T02:58:09.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.613+0000 7fa5aeaea640 1 -- 172.21.15.17:0/1562967577 shutdown_connections 2024-03-24T02:58:09.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.613+0000 7fa5aeaea640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa5980a2800 0x7fa5980a4cc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:09.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.613+0000 7fa5aeaea640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a8102190 0x7fa5a81030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:09.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.613+0000 7fa5aeaea640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a81017d0 0x7fa5a8102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:09.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.613+0000 7fa5aeaea640 1 --2- 172.21.15.17:0/1562967577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a81005e0 0x7fa5a8104570 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:09.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.613+0000 7fa5aeaea640 1 -- 172.21.15.17:0/1562967577 >> 172.21.15.17:0/1562967577 conn(0x7fa5a80fbb50 msgr2=0x7fa5a80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:09.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.613+0000 7fa5aeaea640 1 -- 172.21.15.17:0/1562967577 shutdown_connections 2024-03-24T02:58:09.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:09.613+0000 7fa5aeaea640 1 -- 172.21.15.17:0/1562967577 wait complete. 2024-03-24T02:58:09.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:09 smithi017 ceph-mon[31926]: Deploying daemon mds.e on smithi019 2024-03-24T02:58:09.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:09 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.17:6836/3530669946,v1:172.21.15.17:6837/3530669946] up:boot 2024-03-24T02:58:09.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:09 smithi017 ceph-mon[31926]: fsmap 4 up:standby 2024-03-24T02:58:09.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:09 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-24T02:58:09.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:09 smithi188 ceph-mon[36206]: Deploying daemon mds.e on smithi019 2024-03-24T02:58:09.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:09 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.17:6836/3530669946,v1:172.21.15.17:6837/3530669946] up:boot 2024-03-24T02:58:09.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:09 smithi188 ceph-mon[36206]: fsmap 4 up:standby 2024-03-24T02:58:09.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:09 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-24T02:58:10.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:10 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1562967577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:10.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:10 smithi019 ceph-mon[35944]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:10.571 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 02:58:10 smithi019 systemd[1]: Starting Ceph mds.e for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:58:10.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:10 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1562967577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:10.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:10 smithi017 ceph-mon[31926]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:10.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:10 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1562967577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:10.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:10 smithi188 ceph-mon[36206]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:10.935 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 02:58:10 smithi019 podman[65853]: 2024-03-24 02:58:10.570839415 +0000 UTC m=+0.018773838 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T02:58:10.936 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 02:58:10 smithi019 podman[65853]: 2024-03-24 02:58:10.68278272 +0000 UTC m=+0.130717123 container create 2d477d3fb304776e074101b02ab6e3dad0277da7fb182e8798b21d6f714e55c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-24T02:58:11.320 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 02:58:10 smithi019 podman[65853]: 2024-03-24 02:58:10.936141937 +0000 UTC m=+0.384076340 container init 2d477d3fb304776e074101b02ab6e3dad0277da7fb182e8798b21d6f714e55c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-24T02:58:11.321 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 02:58:10 smithi019 podman[65853]: 2024-03-24 02:58:10.939680591 +0000 UTC m=+0.387615008 container start 2d477d3fb304776e074101b02ab6e3dad0277da7fb182e8798b21d6f714e55c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-24T02:58:11.321 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 02:58:10 smithi019 ceph-mds[65873]: -- 172.21.15.19:0/1063043449 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555a89f6fa40 con 0x555a89f43000 2024-03-24T02:58:11.321 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 02:58:10 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e[65869]: starting mds.e at 2024-03-24T02:58:11.321 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 02:58:11 smithi019 bash[65853]: 2d477d3fb304776e074101b02ab6e3dad0277da7fb182e8798b21d6f714e55c6 2024-03-24T02:58:11.321 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 02:58:11 smithi019 systemd[1]: Started Ceph mds.e for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:58:11.625 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.741+0000 7f390824b640 1 -- 172.21.15.17:0/1599126386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39001017d0 msgr2=0x7f3900101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.741+0000 7f390824b640 1 --2- 172.21.15.17:0/1599126386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39001017d0 0x7f3900101c50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009900 tx=0x7f38f002bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.741+0000 7f390824b640 1 -- 172.21.15.17:0/1599126386 shutdown_connections 2024-03-24T02:58:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.741+0000 7f390824b640 1 --2- 172.21.15.17:0/1599126386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3900102190 0x7f3900108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.741+0000 7f390824b640 1 --2- 172.21.15.17:0/1599126386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39001017d0 0x7f3900101c50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.741+0000 7f390824b640 1 --2- 172.21.15.17:0/1599126386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39001005e0 0x7f3900100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.741+0000 7f390824b640 1 -- 172.21.15.17:0/1599126386 >> 172.21.15.17:0/1599126386 conn(0x7f39000fbb50 msgr2=0x7f39000fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.742+0000 7f390824b640 1 -- 172.21.15.17:0/1599126386 shutdown_connections 2024-03-24T02:58:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.742+0000 7f390824b640 1 -- 172.21.15.17:0/1599126386 wait complete. 2024-03-24T02:58:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.742+0000 7f390824b640 1 Processor -- start 2024-03-24T02:58:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.742+0000 7f390824b640 1 -- start start 2024-03-24T02:58:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.742+0000 7f390824b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39001005e0 0x7f390006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.742+0000 7f390824b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39001017d0 0x7f390019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.742+0000 7f390824b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3900102190 0x7f390019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.742+0000 7f390824b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390010c240 con 0x7f3900102190 2024-03-24T02:58:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.742+0000 7f390824b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390010c0c0 con 0x7f39001017d0 2024-03-24T02:58:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.742+0000 7f390824b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390010c3c0 con 0x7f39001005e0 2024-03-24T02:58:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f39067e8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3900102190 0x7f390019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f39067e8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3900102190 0x7f390019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55458/0 (socket says 172.21.15.17:55458) 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f39067e8640 1 -- 172.21.15.17:0/1696574882 learned_addr learned my addr 172.21.15.17:0/1696574882 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f3905fe7640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39001005e0 0x7f390006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f39057e6640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39001017d0 0x7f390019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f39067e8640 1 -- 172.21.15.17:0/1696574882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39001005e0 msgr2=0x7f390006a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f39067e8640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39001005e0 0x7f390006a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f39067e8640 1 -- 172.21.15.17:0/1696574882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39001017d0 msgr2=0x7f390019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f39067e8640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39001017d0 0x7f390019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f39067e8640 1 -- 172.21.15.17:0/1696574882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390019f630 con 0x7f3900102190 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f39057e6640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39001017d0 0x7f390019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f39067e8640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3900102190 0x7f390019ed20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00eaa0 tx=0x7f38fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f38f6ffd640 1 -- 172.21.15.17:0/1696574882 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00cdb0 con 0x7f3900102190 2024-03-24T02:58:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f390824b640 1 -- 172.21.15.17:0/1696574882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390019f8c0 con 0x7f3900102190 2024-03-24T02:58:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f38f6ffd640 1 -- 172.21.15.17:0/1696574882 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38fc005020 con 0x7f3900102190 2024-03-24T02:58:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f38f6ffd640 1 -- 172.21.15.17:0/1696574882 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0116b0 con 0x7f3900102190 2024-03-24T02:58:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.743+0000 7f390824b640 1 -- 172.21.15.17:0/1696574882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390019fd90 con 0x7f3900102190 2024-03-24T02:58:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.744+0000 7f390824b640 1 -- 172.21.15.17:0/1696574882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3900107950 con 0x7f3900102190 2024-03-24T02:58:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.745+0000 7f38f6ffd640 1 -- 172.21.15.17:0/1696574882 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f38fc005560 con 0x7f3900102190 2024-03-24T02:58:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.746+0000 7f38f6ffd640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f38ec0a2800 0x7f38ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.746+0000 7f38f6ffd640 1 -- 172.21.15.17:0/1696574882 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f38fc0ce820 con 0x7f3900102190 2024-03-24T02:58:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.746+0000 7f3905fe7640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f38ec0a2800 0x7f38ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.746+0000 7f3905fe7640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f38ec0a2800 0x7f38ec0a4cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3900101630 tx=0x7f38f800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.751+0000 7f38f6ffd640 1 -- 172.21.15.17:0/1696574882 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38fc016030 con 0x7f3900102190 2024-03-24T02:58:11.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.973+0000 7f390824b640 1 -- 172.21.15.17:0/1696574882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f390002d070 con 0x7f3900102190 2024-03-24T02:58:11.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.973+0000 7f38f6ffd640 1 -- 172.21.15.17:0/1696574882 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4489 (secure 0 0 0) 0x7f38fc09bd60 con 0x7f3900102190 2024-03-24T02:58:11.974 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:11.974 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-24T02:58:11.974 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 6 2024-03-24T02:58:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.975+0000 7f390824b640 1 -- 172.21.15.17:0/1696574882 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f38ec0a2800 msgr2=0x7f38ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:11.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.975+0000 7f390824b640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f38ec0a2800 0x7f38ec0a4cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3900101630 tx=0x7f38f800b390 comp rx=0 tx=0).stop 2024-03-24T02:58:11.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.975+0000 7f390824b640 1 -- 172.21.15.17:0/1696574882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3900102190 msgr2=0x7f390019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:11.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.975+0000 7f390824b640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3900102190 0x7f390019ed20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00eaa0 tx=0x7f38fc00ef60 comp rx=0 tx=0).stop 2024-03-24T02:58:11.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.975+0000 7f390824b640 1 -- 172.21.15.17:0/1696574882 shutdown_connections 2024-03-24T02:58:11.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.975+0000 7f390824b640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f38ec0a2800 0x7f38ec0a4cc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:11.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.975+0000 7f390824b640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3900102190 0x7f390019ed20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:11.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.975+0000 7f390824b640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39001017d0 0x7f390019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:11.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.975+0000 7f390824b640 1 --2- 172.21.15.17:0/1696574882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39001005e0 0x7f390006a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:11.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.975+0000 7f390824b640 1 -- 172.21.15.17:0/1696574882 >> 172.21.15.17:0/1696574882 conn(0x7f39000fbb50 msgr2=0x7f3900104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:11.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.975+0000 7f390824b640 1 -- 172.21.15.17:0/1696574882 shutdown_connections 2024-03-24T02:58:11.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:11.975+0000 7f390824b640 1 -- 172.21.15.17:0/1696574882 wait complete. 2024-03-24T02:58:12.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:12 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:12.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:12 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:12.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:12 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:12.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:12 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:12.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:12 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:12.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:12 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:12.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:12 smithi017 ceph-mon[31926]: Deploying daemon mds.f on smithi188 2024-03-24T02:58:12.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:12 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.19:6834/1025006466,v1:172.21.15.19:6835/1025006466] up:boot 2024-03-24T02:58:12.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:12 smithi017 ceph-mon[31926]: fsmap 5 up:standby 2024-03-24T02:58:12.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:12 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-24T02:58:12.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:12 smithi017 ceph-mon[31926]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:12.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:12 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1696574882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:12.468 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:12 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:12.468 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:12 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:12.468 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:12 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:12.468 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:12 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:12.468 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:12 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:12.468 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:12 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:12.468 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:12 smithi188 ceph-mon[36206]: Deploying daemon mds.f on smithi188 2024-03-24T02:58:12.468 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:12 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.19:6834/1025006466,v1:172.21.15.19:6835/1025006466] up:boot 2024-03-24T02:58:12.468 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:12 smithi188 ceph-mon[36206]: fsmap 5 up:standby 2024-03-24T02:58:12.469 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:12 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-24T02:58:12.469 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:12 smithi188 ceph-mon[36206]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:12.469 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:12 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1696574882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:12.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:12 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:12.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:12 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:12.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:12 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:12.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:12 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:12.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:12 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:12.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:12 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:12.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:12 smithi019 ceph-mon[35944]: Deploying daemon mds.f on smithi188 2024-03-24T02:58:12.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:12 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.19:6834/1025006466,v1:172.21.15.19:6835/1025006466] up:boot 2024-03-24T02:58:12.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:12 smithi019 ceph-mon[35944]: fsmap 5 up:standby 2024-03-24T02:58:12.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:12 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-24T02:58:12.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:12 smithi019 ceph-mon[35944]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:12.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:12 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1696574882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:13.487 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 02:58:13 smithi188 systemd[1]: Starting Ceph mds.f for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:58:13.750 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 02:58:13 smithi188 podman[66519]: 2024-03-24 02:58:13.391206737 +0000 UTC m=+0.017832987 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T02:58:13.750 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 02:58:13 smithi188 podman[66519]: 2024-03-24 02:58:13.506237389 +0000 UTC m=+0.132863673 container create 8d20f8a515d950eb5b34e75aa2e6fcf08d76972aa7b74283500edddc7a567eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-24T02:58:13.751 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 02:58:13 smithi188 podman[66519]: 2024-03-24 02:58:13.725837431 +0000 UTC m=+0.352463679 container init 8d20f8a515d950eb5b34e75aa2e6fcf08d76972aa7b74283500edddc7a567eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-24T02:58:13.751 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 02:58:13 smithi188 podman[66519]: 2024-03-24 02:58:13.728469376 +0000 UTC m=+0.355095619 container start 8d20f8a515d950eb5b34e75aa2e6fcf08d76972aa7b74283500edddc7a567eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-24T02:58:13.751 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 02:58:13 smithi188 ceph-mds[66539]: -- 172.21.15.188:0/1812900076 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564d8f021a40 con 0x564d8eff4c00 2024-03-24T02:58:13.987 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:14.047 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 02:58:13 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-f[66535]: starting mds.f at 2024-03-24T02:58:14.048 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 02:58:13 smithi188 bash[66519]: 8d20f8a515d950eb5b34e75aa2e6fcf08d76972aa7b74283500edddc7a567eaa 2024-03-24T02:58:14.048 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 02:58:13 smithi188 systemd[1]: Started Ceph mds.f for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:58:14.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.102+0000 7f153b80b640 1 -- 172.21.15.17:0/3333397429 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15341021c0 msgr2=0x7f15341045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.102+0000 7f153b80b640 1 --2- 172.21.15.17:0/3333397429 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15341021c0 0x7f15341045b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1530009900 tx=0x7f153002bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.102+0000 7f153b80b640 1 -- 172.21.15.17:0/3333397429 shutdown_connections 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.102+0000 7f153b80b640 1 --2- 172.21.15.17:0/3333397429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534107420 0x7f1534109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.102+0000 7f153b80b640 1 --2- 172.21.15.17:0/3333397429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1534104af0 0x7f1534106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.102+0000 7f153b80b640 1 --2- 172.21.15.17:0/3333397429 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15341021c0 0x7f15341045b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.102+0000 7f153b80b640 1 -- 172.21.15.17:0/3333397429 >> 172.21.15.17:0/3333397429 conn(0x7f15340fbb50 msgr2=0x7f15340fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.102+0000 7f153b80b640 1 -- 172.21.15.17:0/3333397429 shutdown_connections 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.102+0000 7f153b80b640 1 -- 172.21.15.17:0/3333397429 wait complete. 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.102+0000 7f153b80b640 1 Processor -- start 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.102+0000 7f153b80b640 1 -- start start 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f153b80b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15341021c0 0x7f1534197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f153b80b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1534104af0 0x7f1534197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f153b80b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534107420 0x7f153419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f153b80b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153410d360 con 0x7f15341021c0 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f153b80b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153410d1e0 con 0x7f1534107420 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f153b80b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153410d4e0 con 0x7f1534104af0 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f15395a7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15341021c0 0x7f1534197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f15395a7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15341021c0 0x7f1534197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55476/0 (socket says 172.21.15.17:55476) 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f15395a7640 1 -- 172.21.15.17:0/1739479942 learned_addr learned my addr 172.21.15.17:0/1739479942 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f1538da6640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1534104af0 0x7f1534197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f1539da8640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534107420 0x7f153419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f15395a7640 1 -- 172.21.15.17:0/1739479942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1534104af0 msgr2=0x7f1534197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f15395a7640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1534104af0 0x7f1534197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f15395a7640 1 -- 172.21.15.17:0/1739479942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534107420 msgr2=0x7f153419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f15395a7640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534107420 0x7f153419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f15395a7640 1 -- 172.21.15.17:0/1739479942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153419f680 con 0x7f15341021c0 2024-03-24T02:58:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f1539da8640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534107420 0x7f153419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:14.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f15395a7640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15341021c0 0x7f1534197a60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1530009230 tx=0x7f1530005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:14.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f152a7fc640 1 -- 172.21.15.17:0/1739479942 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530041070 con 0x7f15341021c0 2024-03-24T02:58:14.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f152a7fc640 1 -- 172.21.15.17:0/1739479942 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15300054e0 con 0x7f15341021c0 2024-03-24T02:58:14.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f153b80b640 1 -- 172.21.15.17:0/1739479942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153419f910 con 0x7f15341021c0 2024-03-24T02:58:14.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.103+0000 7f152a7fc640 1 -- 172.21.15.17:0/1739479942 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530034440 con 0x7f15341021c0 2024-03-24T02:58:14.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.104+0000 7f153b80b640 1 -- 172.21.15.17:0/1739479942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153419fe00 con 0x7f15341021c0 2024-03-24T02:58:14.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.104+0000 7f153b80b640 1 -- 172.21.15.17:0/1739479942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15340fd640 con 0x7f15341021c0 2024-03-24T02:58:14.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.105+0000 7f152a7fc640 1 -- 172.21.15.17:0/1739479942 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1530044070 con 0x7f15341021c0 2024-03-24T02:58:14.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.106+0000 7f152a7fc640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f15200a2800 0x7f15200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:14.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.106+0000 7f1538da6640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f15200a2800 0x7f15200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:14.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.106+0000 7f152a7fc640 1 -- 172.21.15.17:0/1739479942 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f15300f0740 con 0x7f15341021c0 2024-03-24T02:58:14.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.107+0000 7f1538da6640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f15200a2800 0x7f15200a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1534198f70 tx=0x7f152400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:14.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.110+0000 7f152a7fc640 1 -- 172.21.15.17:0/1739479942 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f15300bdbe0 con 0x7f15341021c0 2024-03-24T02:58:14.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.337+0000 7f153b80b640 1 -- 172.21.15.17:0/1739479942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f153402d270 con 0x7f15341021c0 2024-03-24T02:58:14.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.338+0000 7f152a7fc640 1 -- 172.21.15.17:0/1739479942 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5283 (secure 0 0 0) 0x7f15300bd4e0 con 0x7f15341021c0 2024-03-24T02:58:14.338 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:14.338 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-24T02:58:14.338 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2024-03-24T02:58:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.340+0000 7f153b80b640 1 -- 172.21.15.17:0/1739479942 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f15200a2800 msgr2=0x7f15200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.340+0000 7f153b80b640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f15200a2800 0x7f15200a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1534198f70 tx=0x7f152400b410 comp rx=0 tx=0).stop 2024-03-24T02:58:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.340+0000 7f153b80b640 1 -- 172.21.15.17:0/1739479942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15341021c0 msgr2=0x7f1534197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.340+0000 7f153b80b640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15341021c0 0x7f1534197a60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1530009230 tx=0x7f1530005dd0 comp rx=0 tx=0).stop 2024-03-24T02:58:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.340+0000 7f153b80b640 1 -- 172.21.15.17:0/1739479942 shutdown_connections 2024-03-24T02:58:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.340+0000 7f153b80b640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f15200a2800 0x7f15200a4cc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.340+0000 7f153b80b640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534107420 0x7f153419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.340+0000 7f153b80b640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1534104af0 0x7f1534197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.340+0000 7f153b80b640 1 --2- 172.21.15.17:0/1739479942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15341021c0 0x7f1534197a60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.340+0000 7f153b80b640 1 -- 172.21.15.17:0/1739479942 >> 172.21.15.17:0/1739479942 conn(0x7f15340fbb50 msgr2=0x7f1534102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.340+0000 7f153b80b640 1 -- 172.21.15.17:0/1739479942 shutdown_connections 2024-03-24T02:58:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:14.340+0000 7f153b80b640 1 -- 172.21.15.17:0/1739479942 wait complete. 2024-03-24T02:58:15.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:14 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] up:boot 2024-03-24T02:58:15.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:14 smithi019 ceph-mon[35944]: fsmap 6 up:standby 2024-03-24T02:58:15.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:14 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-24T02:58:15.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:14 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:15.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:14 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:15.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:14 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:15.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:14 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:15.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:14 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:15.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:14 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:15.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:14 smithi019 ceph-mon[35944]: Deploying daemon mds.c on smithi188 2024-03-24T02:58:15.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:14 smithi019 ceph-mon[35944]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:15.144 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:14 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1739479942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:15.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:14 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] up:boot 2024-03-24T02:58:15.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:14 smithi017 ceph-mon[31926]: fsmap 6 up:standby 2024-03-24T02:58:15.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:14 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-24T02:58:15.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:14 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:15.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:14 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:15.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:14 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:15.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:14 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:15.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:14 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:15.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:14 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:15.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:14 smithi017 ceph-mon[31926]: Deploying daemon mds.c on smithi188 2024-03-24T02:58:15.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:14 smithi017 ceph-mon[31926]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:15.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:14 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1739479942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:15.283 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:14 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] up:boot 2024-03-24T02:58:15.283 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:14 smithi188 ceph-mon[36206]: fsmap 6 up:standby 2024-03-24T02:58:15.283 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:14 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-24T02:58:15.283 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:14 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:15.283 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:14 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:15.283 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:14 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:15.283 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:14 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:15.284 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:14 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:15.284 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:14 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:15.284 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:14 smithi188 ceph-mon[36206]: Deploying daemon mds.c on smithi188 2024-03-24T02:58:15.284 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:14 smithi188 ceph-mon[36206]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:15.284 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:14 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1739479942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:16.150 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:16 smithi188 ceph-mon[36206]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:16.151 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 02:58:16 smithi188 systemd[1]: Starting Ceph mds.c for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:58:16.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:16 smithi019 ceph-mon[35944]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:16.352 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:16.380 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:16 smithi017 ceph-mon[31926]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.467+0000 7f29b31a9640 1 -- 172.21.15.17:0/1643767360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29ac101830 msgr2=0x7f29ac101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.467+0000 7f29b31a9640 1 --2- 172.21.15.17:0/1643767360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29ac101830 0x7f29ac101cb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2994009900 tx=0x7f299402bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.467+0000 7f29b31a9640 1 -- 172.21.15.17:0/1643767360 shutdown_connections 2024-03-24T02:58:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.467+0000 7f29b31a9640 1 --2- 172.21.15.17:0/1643767360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29ac1021f0 0x7f29ac108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.467+0000 7f29b31a9640 1 --2- 172.21.15.17:0/1643767360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29ac101830 0x7f29ac101cb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.467+0000 7f29b31a9640 1 --2- 172.21.15.17:0/1643767360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29ac100640 0x7f29ac100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.467+0000 7f29b31a9640 1 -- 172.21.15.17:0/1643767360 >> 172.21.15.17:0/1643767360 conn(0x7f29ac0fbbd0 msgr2=0x7f29ac0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.467+0000 7f29b31a9640 1 -- 172.21.15.17:0/1643767360 shutdown_connections 2024-03-24T02:58:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.467+0000 7f29b31a9640 1 -- 172.21.15.17:0/1643767360 wait complete. 2024-03-24T02:58:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b31a9640 1 Processor -- start 2024-03-24T02:58:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b31a9640 1 -- start start 2024-03-24T02:58:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b31a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29ac100640 0x7f29ac06a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b31a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29ac101830 0x7f29ac19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b31a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29ac1021f0 0x7f29ac19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b31a9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac10c230 con 0x7f29ac100640 2024-03-24T02:58:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b31a9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac10c0b0 con 0x7f29ac101830 2024-03-24T02:58:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b31a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac10c3b0 con 0x7f29ac1021f0 2024-03-24T02:58:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b0f45640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29ac100640 0x7f29ac06a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b0f45640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29ac100640 0x7f29ac06a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55488/0 (socket says 172.21.15.17:55488) 2024-03-24T02:58:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b0f45640 1 -- 172.21.15.17:0/333436631 learned_addr learned my addr 172.21.15.17:0/333436631 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29abfff640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29ac101830 0x7f29ac19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b1746640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29ac1021f0 0x7f29ac19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b0f45640 1 -- 172.21.15.17:0/333436631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29ac1021f0 msgr2=0x7f29ac19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b0f45640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29ac1021f0 0x7f29ac19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b0f45640 1 -- 172.21.15.17:0/333436631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29ac101830 msgr2=0x7f29ac19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b0f45640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29ac101830 0x7f29ac19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.468+0000 7f29b0f45640 1 -- 172.21.15.17:0/333436631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac19f5d0 con 0x7f29ac100640 2024-03-24T02:58:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.469+0000 7f29b0f45640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29ac100640 0x7f29ac06a860 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f29a400ea80 tx=0x7f29a400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.469+0000 7f29a9ffb640 1 -- 172.21.15.17:0/333436631 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a400ce40 con 0x7f29ac100640 2024-03-24T02:58:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.469+0000 7f29a9ffb640 1 -- 172.21.15.17:0/333436631 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29a4011950 con 0x7f29ac100640 2024-03-24T02:58:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.469+0000 7f29b31a9640 1 -- 172.21.15.17:0/333436631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ac19f8c0 con 0x7f29ac100640 2024-03-24T02:58:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.469+0000 7f29a9ffb640 1 -- 172.21.15.17:0/333436631 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a401ab80 con 0x7f29ac100640 2024-03-24T02:58:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.469+0000 7f29b31a9640 1 -- 172.21.15.17:0/333436631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ac19fde0 con 0x7f29ac100640 2024-03-24T02:58:16.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.470+0000 7f29b31a9640 1 -- 172.21.15.17:0/333436631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ac1079b0 con 0x7f29ac100640 2024-03-24T02:58:16.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.471+0000 7f29a9ffb640 1 -- 172.21.15.17:0/333436631 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f29a4011b00 con 0x7f29ac100640 2024-03-24T02:58:16.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.471+0000 7f29a9ffb640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f29980a2800 0x7f29980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:16.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.472+0000 7f29abfff640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f29980a2800 0x7f29980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:16.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.472+0000 7f29a9ffb640 1 -- 172.21.15.17:0/333436631 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f29a40ceb90 con 0x7f29ac100640 2024-03-24T02:58:16.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.472+0000 7f29abfff640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f29980a2800 0x7f29980a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f299402c7b0 tx=0x7f2994005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:16.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.476+0000 7f29a9ffb640 1 -- 172.21.15.17:0/333436631 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f29a4016030 con 0x7f29ac100640 2024-03-24T02:58:16.497 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 02:58:16 smithi188 podman[66823]: 2024-03-24 02:58:16.245197473 +0000 UTC m=+0.018189869 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T02:58:16.497 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 02:58:16 smithi188 podman[66823]: 2024-03-24 02:58:16.370988102 +0000 UTC m=+0.143980482 container create aab8aa032050a71cc7afb5bd6867ffe5defda82cb11ed42b2c6445b87997da1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-24T02:58:16.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.699+0000 7f29b31a9640 1 -- 172.21.15.17:0/333436631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29ac02d070 con 0x7f29ac100640 2024-03-24T02:58:16.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.700+0000 7f29a9ffb640 1 -- 172.21.15.17:0/333436631 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5283 (secure 0 0 0) 0x7f29a409c0d0 con 0x7f29ac100640 2024-03-24T02:58:16.700 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:16.700 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-24T02:58:16.700 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2024-03-24T02:58:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.701+0000 7f29b31a9640 1 -- 172.21.15.17:0/333436631 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f29980a2800 msgr2=0x7f29980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.701+0000 7f29b31a9640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f29980a2800 0x7f29980a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f299402c7b0 tx=0x7f2994005fd0 comp rx=0 tx=0).stop 2024-03-24T02:58:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.702+0000 7f29b31a9640 1 -- 172.21.15.17:0/333436631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29ac100640 msgr2=0x7f29ac06a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.702+0000 7f29b31a9640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29ac100640 0x7f29ac06a860 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f29a400ea80 tx=0x7f29a400ef40 comp rx=0 tx=0).stop 2024-03-24T02:58:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.702+0000 7f29b31a9640 1 -- 172.21.15.17:0/333436631 shutdown_connections 2024-03-24T02:58:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.702+0000 7f29b31a9640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f29980a2800 0x7f29980a4cc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.702+0000 7f29b31a9640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29ac1021f0 0x7f29ac19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.702+0000 7f29b31a9640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29ac101830 0x7f29ac19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:16.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.702+0000 7f29b31a9640 1 --2- 172.21.15.17:0/333436631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29ac100640 0x7f29ac06a860 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:16.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.702+0000 7f29b31a9640 1 -- 172.21.15.17:0/333436631 >> 172.21.15.17:0/333436631 conn(0x7f29ac0fbbd0 msgr2=0x7f29ac104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:16.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.702+0000 7f29b31a9640 1 -- 172.21.15.17:0/333436631 shutdown_connections 2024-03-24T02:58:16.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:16.702+0000 7f29b31a9640 1 -- 172.21.15.17:0/333436631 wait complete. 2024-03-24T02:58:16.797 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 02:58:16 smithi188 podman[66823]: 2024-03-24 02:58:16.607311885 +0000 UTC m=+0.380304291 container init aab8aa032050a71cc7afb5bd6867ffe5defda82cb11ed42b2c6445b87997da1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T02:58:16.798 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 02:58:16 smithi188 podman[66823]: 2024-03-24 02:58:16.610854142 +0000 UTC m=+0.383846518 container start aab8aa032050a71cc7afb5bd6867ffe5defda82cb11ed42b2c6445b87997da1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-24T02:58:16.798 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 02:58:16 smithi188 ceph-mds[66852]: -- 172.21.15.188:0/85276778 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5639f05d3a40 con 0x5639f05a6c00 2024-03-24T02:58:16.798 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 02:58:16 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-c[66846]: starting mds.c at 2024-03-24T02:58:16.798 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 02:58:16 smithi188 bash[66823]: aab8aa032050a71cc7afb5bd6867ffe5defda82cb11ed42b2c6445b87997da1e 2024-03-24T02:58:16.798 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 02:58:16 smithi188 systemd[1]: Started Ceph mds.c for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:58:17.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:17 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/333436631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:17.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:17.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:17.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:17.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:17.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:17.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:17 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:17.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:17 smithi188 ceph-mon[36206]: Deploying daemon mds.h on smithi019 2024-03-24T02:58:17.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:17 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/333436631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:17.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:17.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:17.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:17.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:17.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:17.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:17 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:17.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:17 smithi019 ceph-mon[35944]: Deploying daemon mds.h on smithi019 2024-03-24T02:58:17.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:17 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/333436631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:17.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:17.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:17.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:17.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:17.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:17.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:17 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:17.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:17 smithi017 ceph-mon[31926]: Deploying daemon mds.h on smithi019 2024-03-24T02:58:18.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:18 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] up:boot 2024-03-24T02:58:18.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:18 smithi188 ceph-mon[36206]: fsmap 7 up:standby 2024-03-24T02:58:18.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:18 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-24T02:58:18.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:18 smithi188 ceph-mon[36206]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:18 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] up:boot 2024-03-24T02:58:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:18 smithi019 ceph-mon[35944]: fsmap 7 up:standby 2024-03-24T02:58:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:18 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-24T02:58:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:18 smithi019 ceph-mon[35944]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:18.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:18 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] up:boot 2024-03-24T02:58:18.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:18 smithi017 ceph-mon[31926]: fsmap 7 up:standby 2024-03-24T02:58:18.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:18 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-24T02:58:18.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:18 smithi017 ceph-mon[31926]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:18.714 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:18.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.829+0000 7fbc05820640 1 -- 172.21.15.17:0/1341500873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc00103270 msgr2=0x7fbc00109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:18.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.829+0000 7fbc05820640 1 --2- 172.21.15.17:0/1341500873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc00103270 0x7fbc00109b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009ca0 tx=0x7fbbf002c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:18.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.829+0000 7fbc05820640 1 -- 172.21.15.17:0/1341500873 shutdown_connections 2024-03-24T02:58:18.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.829+0000 7fbc05820640 1 --2- 172.21.15.17:0/1341500873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc00103270 0x7fbc00109b00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:18.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.829+0000 7fbc05820640 1 --2- 172.21.15.17:0/1341500873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc001028b0 0x7fbc00102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:18.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.829+0000 7fbc05820640 1 --2- 172.21.15.17:0/1341500873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc001016c0 0x7fbc00101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:18.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.829+0000 7fbc05820640 1 -- 172.21.15.17:0/1341500873 >> 172.21.15.17:0/1341500873 conn(0x7fbc000fcc50 msgr2=0x7fbc000ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:18.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.829+0000 7fbc05820640 1 -- 172.21.15.17:0/1341500873 shutdown_connections 2024-03-24T02:58:18.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.829+0000 7fbc05820640 1 -- 172.21.15.17:0/1341500873 wait complete. 2024-03-24T02:58:18.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.829+0000 7fbc05820640 1 Processor -- start 2024-03-24T02:58:18.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.829+0000 7fbc05820640 1 -- start start 2024-03-24T02:58:18.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbc05820640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc001016c0 0x7fbc001979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:18.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbc05820640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc001028b0 0x7fbc00197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:18.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbc05820640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc00103270 0x7fbc0019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:18.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbc05820640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0010d320 con 0x7fbc00103270 2024-03-24T02:58:18.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbc05820640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0010d1a0 con 0x7fbc001028b0 2024-03-24T02:58:18.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbc05820640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0010d4a0 con 0x7fbc001016c0 2024-03-24T02:58:18.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbbff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc00103270 0x7fbc0019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbbff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc00103270 0x7fbc0019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55500/0 (socket says 172.21.15.17:55500) 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbbff7fe640 1 -- 172.21.15.17:0/3366471510 learned_addr learned my addr 172.21.15.17:0/3366471510 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbbfe7fc640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc001028b0 0x7fbc00197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbbfeffd640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc001016c0 0x7fbc001979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbbff7fe640 1 -- 172.21.15.17:0/3366471510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc001016c0 msgr2=0x7fbc001979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbbff7fe640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc001016c0 0x7fbc001979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbbff7fe640 1 -- 172.21.15.17:0/3366471510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc001028b0 msgr2=0x7fbc00197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbbff7fe640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc001028b0 0x7fbc00197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbbff7fe640 1 -- 172.21.15.17:0/3366471510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0019f5f0 con 0x7fbc00103270 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbbfeffd640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc001016c0 0x7fbc001979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbbff7fe640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc00103270 0x7fbc0019ef80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0032530 tx=0x7fbbf0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbc0481e640 1 -- 172.21.15.17:0/3366471510 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0041020 con 0x7fbc00103270 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbc0481e640 1 -- 172.21.15.17:0/3366471510 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbf0034cd0 con 0x7fbc00103270 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbc05820640 1 -- 172.21.15.17:0/3366471510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0019f880 con 0x7fbc00103270 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbc0481e640 1 -- 172.21.15.17:0/3366471510 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0039040 con 0x7fbc00103270 2024-03-24T02:58:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.830+0000 7fbc05820640 1 -- 172.21.15.17:0/3366471510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0019fd40 con 0x7fbc00103270 2024-03-24T02:58:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.831+0000 7fbc05820640 1 -- 172.21.15.17:0/3366471510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc00108980 con 0x7fbc00103270 2024-03-24T02:58:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.832+0000 7fbc0481e640 1 -- 172.21.15.17:0/3366471510 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbbf0044020 con 0x7fbc00103270 2024-03-24T02:58:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.833+0000 7fbc0481e640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbbec0a2800 0x7fbbec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.833+0000 7fbbfeffd640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbbec0a2800 0x7fbbec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.833+0000 7fbc0481e640 1 -- 172.21.15.17:0/3366471510 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbbf00f08b0 con 0x7fbc00103270 2024-03-24T02:58:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.834+0000 7fbbfeffd640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbbec0a2800 0x7fbbec0a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbc00102710 tx=0x7fbbf8005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:18.838+0000 7fbc0481e640 1 -- 172.21.15.17:0/3366471510 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbbf004b020 con 0x7fbc00103270 2024-03-24T02:58:19.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.059+0000 7fbc05820640 1 -- 172.21.15.17:0/3366471510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc0002d230 con 0x7fbc00103270 2024-03-24T02:58:19.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.060+0000 7fbc0481e640 1 -- 172.21.15.17:0/3366471510 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6077 (secure 0 0 0) 0x7fbbf00bde80 con 0x7fbc00103270 2024-03-24T02:58:19.060 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:19.060 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-24T02:58:19.060 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 8 2024-03-24T02:58:19.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.062+0000 7fbc05820640 1 -- 172.21.15.17:0/3366471510 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbbec0a2800 msgr2=0x7fbbec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:19.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.062+0000 7fbc05820640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbbec0a2800 0x7fbbec0a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbc00102710 tx=0x7fbbf8005d30 comp rx=0 tx=0).stop 2024-03-24T02:58:19.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.062+0000 7fbc05820640 1 -- 172.21.15.17:0/3366471510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc00103270 msgr2=0x7fbc0019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:19.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.062+0000 7fbc05820640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc00103270 0x7fbc0019ef80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0032530 tx=0x7fbbf0032f40 comp rx=0 tx=0).stop 2024-03-24T02:58:19.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.062+0000 7fbc05820640 1 -- 172.21.15.17:0/3366471510 shutdown_connections 2024-03-24T02:58:19.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.062+0000 7fbc05820640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbbec0a2800 0x7fbbec0a4cc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:19.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.062+0000 7fbc05820640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc00103270 0x7fbc0019ef80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:19.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.062+0000 7fbc05820640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc001028b0 0x7fbc00197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:19.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.062+0000 7fbc05820640 1 --2- 172.21.15.17:0/3366471510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc001016c0 0x7fbc001979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:19.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.062+0000 7fbc05820640 1 -- 172.21.15.17:0/3366471510 >> 172.21.15.17:0/3366471510 conn(0x7fbc000fcc50 msgr2=0x7fbc00105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:19.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.062+0000 7fbc05820640 1 -- 172.21.15.17:0/3366471510 shutdown_connections 2024-03-24T02:58:19.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:19.062+0000 7fbc05820640 1 -- 172.21.15.17:0/3366471510 wait complete. 2024-03-24T02:58:19.113 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 02:58:19 smithi019 systemd[1]: Starting Ceph mds.h for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:58:19.399 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:19 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3366471510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:19.399 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 02:58:19 smithi019 podman[66170]: 2024-03-24 02:58:19.169609346 +0000 UTC m=+0.018704186 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T02:58:19.399 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 02:58:19 smithi019 podman[66170]: 2024-03-24 02:58:19.280085991 +0000 UTC m=+0.129180834 container create 35e3e6dbc206d3a6b4a492926ba08d1b90c25e6c800ca5367e03b7a04683f188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T02:58:19.455 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:19 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3366471510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:19.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:19 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3366471510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:19.665 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 02:58:19 smithi019 podman[66170]: 2024-03-24 02:58:19.532668445 +0000 UTC m=+0.381763300 container init 35e3e6dbc206d3a6b4a492926ba08d1b90c25e6c800ca5367e03b7a04683f188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-24T02:58:19.665 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 02:58:19 smithi019 podman[66170]: 2024-03-24 02:58:19.535359683 +0000 UTC m=+0.384454520 container start 35e3e6dbc206d3a6b4a492926ba08d1b90c25e6c800ca5367e03b7a04683f188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T02:58:19.665 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 02:58:19 smithi019 ceph-mds[66188]: -- 172.21.15.19:0/915532204 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a3bcbd1a40 con 0x55a3bcba5000 2024-03-24T02:58:19.665 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 02:58:19 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h[66184]: starting mds.h at 2024-03-24T02:58:20.070 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 02:58:19 smithi019 bash[66170]: 35e3e6dbc206d3a6b4a492926ba08d1b90c25e6c800ca5367e03b7a04683f188 2024-03-24T02:58:20.070 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 02:58:19 smithi019 systemd[1]: Started Ceph mds.h for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:58:21.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:21.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:21.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:21.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:21.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:21.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:21.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:20 smithi188 ceph-mon[36206]: Deploying daemon mds.i on smithi188 2024-03-24T02:58:21.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:20 smithi188 ceph-mon[36206]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:21.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:20 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] up:boot 2024-03-24T02:58:21.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:20 smithi188 ceph-mon[36206]: fsmap 8 up:standby 2024-03-24T02:58:21.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:20 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-24T02:58:21.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:21.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:21.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:21.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:21.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:21.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:21.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:20 smithi019 ceph-mon[35944]: Deploying daemon mds.i on smithi188 2024-03-24T02:58:21.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:20 smithi019 ceph-mon[35944]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:21.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:20 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] up:boot 2024-03-24T02:58:21.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:20 smithi019 ceph-mon[35944]: fsmap 8 up:standby 2024-03-24T02:58:21.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:20 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-24T02:58:21.074 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:21.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:21.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:21.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:21.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:21.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:21.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:21.104 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:20 smithi017 ceph-mon[31926]: Deploying daemon mds.i on smithi188 2024-03-24T02:58:21.104 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:20 smithi017 ceph-mon[31926]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:21.104 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:20 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] up:boot 2024-03-24T02:58:21.104 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:20 smithi017 ceph-mon[31926]: fsmap 8 up:standby 2024-03-24T02:58:21.104 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:20 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-24T02:58:21.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.191+0000 7fb71d2b2640 1 -- 172.21.15.17:0/2769821926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb718103270 msgr2=0x7fb718109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:21.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.191+0000 7fb71d2b2640 1 --2- 172.21.15.17:0/2769821926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb718103270 0x7fb718109b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb708009ca0 tx=0x7fb70802c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:21.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.191+0000 7fb71d2b2640 1 -- 172.21.15.17:0/2769821926 shutdown_connections 2024-03-24T02:58:21.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.191+0000 7fb71d2b2640 1 --2- 172.21.15.17:0/2769821926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb718103270 0x7fb718109b00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:21.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.191+0000 7fb71d2b2640 1 --2- 172.21.15.17:0/2769821926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7181028b0 0x7fb718102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.191+0000 7fb71d2b2640 1 --2- 172.21.15.17:0/2769821926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7181016c0 0x7fb718101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.191+0000 7fb71d2b2640 1 -- 172.21.15.17:0/2769821926 >> 172.21.15.17:0/2769821926 conn(0x7fb7180fcc50 msgr2=0x7fb7180ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.191+0000 7fb71d2b2640 1 -- 172.21.15.17:0/2769821926 shutdown_connections 2024-03-24T02:58:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.191+0000 7fb71d2b2640 1 -- 172.21.15.17:0/2769821926 wait complete. 2024-03-24T02:58:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.191+0000 7fb71d2b2640 1 Processor -- start 2024-03-24T02:58:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.191+0000 7fb71d2b2640 1 -- start start 2024-03-24T02:58:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb71d2b2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7181016c0 0x7fb7181979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb71d2b2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7181028b0 0x7fb718197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb71d2b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb718103270 0x7fb71819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb71d2b2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71810d320 con 0x7fb7181028b0 2024-03-24T02:58:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb71d2b2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71810d1a0 con 0x7fb718103270 2024-03-24T02:58:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb71d2b2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71810d4a0 con 0x7fb7181016c0 2024-03-24T02:58:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb7167fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7181028b0 0x7fb718197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb7167fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7181028b0 0x7fb718197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57822/0 (socket says 172.21.15.17:57822) 2024-03-24T02:58:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb7167fc640 1 -- 172.21.15.17:0/1671245221 learned_addr learned my addr 172.21.15.17:0/1671245221 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb716ffd640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7181016c0 0x7fb7181979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb7177fe640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb718103270 0x7fb71819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb7167fc640 1 -- 172.21.15.17:0/1671245221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7181016c0 msgr2=0x7fb7181979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb7167fc640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7181016c0 0x7fb7181979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb7167fc640 1 -- 172.21.15.17:0/1671245221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb718103270 msgr2=0x7fb71819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb7167fc640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb718103270 0x7fb71819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb7167fc640 1 -- 172.21.15.17:0/1671245221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71819f5f0 con 0x7fb7181028b0 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb7177fe640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb718103270 0x7fb71819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb7167fc640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7181028b0 0x7fb718197f10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00c890 tx=0x7fb70c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.192+0000 7fb6fffff640 1 -- 172.21.15.17:0/1671245221 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c004220 con 0x7fb7181028b0 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.193+0000 7fb71d2b2640 1 -- 172.21.15.17:0/1671245221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71819f8e0 con 0x7fb7181028b0 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.193+0000 7fb6fffff640 1 -- 172.21.15.17:0/1671245221 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb70c0043c0 con 0x7fb7181028b0 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.193+0000 7fb6fffff640 1 -- 172.21.15.17:0/1671245221 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c004a00 con 0x7fb7181028b0 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.193+0000 7fb71d2b2640 1 -- 172.21.15.17:0/1671245221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71819fe30 con 0x7fb7181028b0 2024-03-24T02:58:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.194+0000 7fb71d2b2640 1 -- 172.21.15.17:0/1671245221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb718108980 con 0x7fb7181028b0 2024-03-24T02:58:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.195+0000 7fb6fffff640 1 -- 172.21.15.17:0/1671245221 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb70c0079b0 con 0x7fb7181028b0 2024-03-24T02:58:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.195+0000 7fb6fffff640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb7000a2800 0x7fb7000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.195+0000 7fb716ffd640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb7000a2800 0x7fb7000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.195+0000 7fb6fffff640 1 -- 172.21.15.17:0/1671245221 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb70c0ce250 con 0x7fb7181028b0 2024-03-24T02:58:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.196+0000 7fb716ffd640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb7000a2800 0x7fb7000a4cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb710005fd0 tx=0x7fb71000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:21.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.200+0000 7fb6fffff640 1 -- 172.21.15.17:0/1671245221 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb70c014030 con 0x7fb7181028b0 2024-03-24T02:58:21.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.423+0000 7fb71d2b2640 1 -- 172.21.15.17:0/1671245221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb71802d2d0 con 0x7fb7181028b0 2024-03-24T02:58:21.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.424+0000 7fb6fffff640 1 -- 172.21.15.17:0/1671245221 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6868 (secure 0 0 0) 0x7fb70c09b790 con 0x7fb7181028b0 2024-03-24T02:58:21.423 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:21.423 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-24T02:58:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 9 2024-03-24T02:58:21.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.426+0000 7fb71d2b2640 1 -- 172.21.15.17:0/1671245221 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb7000a2800 msgr2=0x7fb7000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:21.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.426+0000 7fb71d2b2640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb7000a2800 0x7fb7000a4cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb710005fd0 tx=0x7fb71000b410 comp rx=0 tx=0).stop 2024-03-24T02:58:21.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.426+0000 7fb71d2b2640 1 -- 172.21.15.17:0/1671245221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7181028b0 msgr2=0x7fb718197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:21.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.426+0000 7fb71d2b2640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7181028b0 0x7fb718197f10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00c890 tx=0x7fb70c00cd50 comp rx=0 tx=0).stop 2024-03-24T02:58:21.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.426+0000 7fb71d2b2640 1 -- 172.21.15.17:0/1671245221 shutdown_connections 2024-03-24T02:58:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.426+0000 7fb71d2b2640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb7000a2800 0x7fb7000a4cc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.426+0000 7fb71d2b2640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb718103270 0x7fb71819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.426+0000 7fb71d2b2640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7181028b0 0x7fb718197f10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.426+0000 7fb71d2b2640 1 --2- 172.21.15.17:0/1671245221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7181016c0 0x7fb7181979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.426+0000 7fb71d2b2640 1 -- 172.21.15.17:0/1671245221 >> 172.21.15.17:0/1671245221 conn(0x7fb7180fcc50 msgr2=0x7fb718105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.426+0000 7fb71d2b2640 1 -- 172.21.15.17:0/1671245221 shutdown_connections 2024-03-24T02:58:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:21.426+0000 7fb71d2b2640 1 -- 172.21.15.17:0/1671245221 wait complete. 2024-03-24T02:58:22.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:21 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1671245221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:22.048 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:58:21 smithi188 systemd[1]: Starting Ceph mds.i for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:58:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:21 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1671245221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:21 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1671245221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:22.371 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:58:22 smithi188 podman[67141]: 2024-03-24 02:58:22.000494598 +0000 UTC m=+0.017584281 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T02:58:22.371 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:58:22 smithi188 podman[67141]: 2024-03-24 02:58:22.102214795 +0000 UTC m=+0.119304484 container create f3ce1e99fefa3041537e9646cb74de250e2b728a3543acb85bd79f7891d13bbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-03-24T02:58:22.371 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:58:22 smithi188 podman[67141]: 2024-03-24 02:58:22.371990615 +0000 UTC m=+0.389080278 container init f3ce1e99fefa3041537e9646cb74de250e2b728a3543acb85bd79f7891d13bbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-24T02:58:22.742 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:58:22 smithi188 podman[67141]: 2024-03-24 02:58:22.374762348 +0000 UTC m=+0.391852010 container start f3ce1e99fefa3041537e9646cb74de250e2b728a3543acb85bd79f7891d13bbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T02:58:22.742 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:58:22 smithi188 ceph-mds[67160]: -- 172.21.15.188:0/807120659 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621302d5a40 con 0x5621302a8c00 2024-03-24T02:58:22.742 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:58:22 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i[67156]: starting mds.i at 2024-03-24T02:58:22.742 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:58:22 smithi188 bash[67141]: f3ce1e99fefa3041537e9646cb74de250e2b728a3543acb85bd79f7891d13bbf 2024-03-24T02:58:22.742 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:58:22 smithi188 systemd[1]: Started Ceph mds.i for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:58:23.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:22 smithi188 ceph-mon[36206]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:23.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:22 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:23.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:22 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:23.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:22 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:23.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:22 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:23.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:22 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:23.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:22 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:23.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:22 smithi019 ceph-mon[35944]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:23.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:22 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:23.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:22 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:23.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:22 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:23.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:22 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:23.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:22 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:23.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:22 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:23.200 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:22 smithi017 ceph-mon[31926]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:23.200 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:22 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:23.200 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:22 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:23.200 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:22 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:23.200 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:22 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:23.200 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:22 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:23.200 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:22 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:23.437 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:23.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.547+0000 7f5392510640 1 -- 172.21.15.17:0/1187891447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f538c103270 msgr2=0x7f538c109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:23.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.547+0000 7f5392510640 1 --2- 172.21.15.17:0/1187891447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f538c103270 0x7f538c109b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5384009ca0 tx=0x7f538402c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:23.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.547+0000 7f5392510640 1 -- 172.21.15.17:0/1187891447 shutdown_connections 2024-03-24T02:58:23.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.547+0000 7f5392510640 1 --2- 172.21.15.17:0/1187891447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f538c103270 0x7f538c109b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:23.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.547+0000 7f5392510640 1 --2- 172.21.15.17:0/1187891447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f538c1028b0 0x7f538c102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:23.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.547+0000 7f5392510640 1 --2- 172.21.15.17:0/1187891447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f538c1016c0 0x7f538c101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:23.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.547+0000 7f5392510640 1 -- 172.21.15.17:0/1187891447 >> 172.21.15.17:0/1187891447 conn(0x7f538c0fcc50 msgr2=0x7f538c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:23.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.547+0000 7f5392510640 1 -- 172.21.15.17:0/1187891447 shutdown_connections 2024-03-24T02:58:23.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.547+0000 7f5392510640 1 -- 172.21.15.17:0/1187891447 wait complete. 2024-03-24T02:58:23.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f5392510640 1 Processor -- start 2024-03-24T02:58:23.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f5392510640 1 -- start start 2024-03-24T02:58:23.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f5392510640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f538c1016c0 0x7f538c197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:23.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f5392510640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f538c1028b0 0x7f538c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:23.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f5392510640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f538c103270 0x7f538c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:23.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f5392510640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c10d340 con 0x7f538c1028b0 2024-03-24T02:58:23.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f5392510640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c10d1c0 con 0x7f538c103270 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f5392510640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c10d4c0 con 0x7f538c1016c0 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f538b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f538c1028b0 0x7f538c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f538b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f538c1028b0 0x7f538c197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57840/0 (socket says 172.21.15.17:57840) 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f538b7fe640 1 -- 172.21.15.17:0/2071644688 learned_addr learned my addr 172.21.15.17:0/2071644688 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f538bfff640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f538c1016c0 0x7f538c197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f5390aad640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f538c103270 0x7f538c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f538b7fe640 1 -- 172.21.15.17:0/2071644688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f538c1016c0 msgr2=0x7f538c197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f538b7fe640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f538c1016c0 0x7f538c197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f538b7fe640 1 -- 172.21.15.17:0/2071644688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f538c103270 msgr2=0x7f538c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f538b7fe640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f538c103270 0x7f538c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.548+0000 7f538b7fe640 1 -- 172.21.15.17:0/2071644688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c19f580 con 0x7f538c1028b0 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.549+0000 7f538b7fe640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f538c1028b0 0x7f538c197ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f538000d950 tx=0x7f538000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.549+0000 7f53897fa640 1 -- 172.21.15.17:0/2071644688 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380017070 con 0x7f538c1028b0 2024-03-24T02:58:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.549+0000 7f5392510640 1 -- 172.21.15.17:0/2071644688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c19f870 con 0x7f538c1028b0 2024-03-24T02:58:23.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.549+0000 7f53897fa640 1 -- 172.21.15.17:0/2071644688 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5380009c30 con 0x7f538c1028b0 2024-03-24T02:58:23.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.549+0000 7f53897fa640 1 -- 172.21.15.17:0/2071644688 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538000f6b0 con 0x7f538c1028b0 2024-03-24T02:58:23.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.549+0000 7f5392510640 1 -- 172.21.15.17:0/2071644688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c19fe10 con 0x7f538c1028b0 2024-03-24T02:58:23.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.550+0000 7f5392510640 1 -- 172.21.15.17:0/2071644688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538c1089e0 con 0x7f538c1028b0 2024-03-24T02:58:23.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.551+0000 7f53897fa640 1 -- 172.21.15.17:0/2071644688 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f53800109d0 con 0x7f538c1028b0 2024-03-24T02:58:23.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.551+0000 7f53897fa640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f53740a2800 0x7f53740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:23.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.551+0000 7f538bfff640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f53740a2800 0x7f53740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:23.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.551+0000 7f53897fa640 1 -- 172.21.15.17:0/2071644688 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f53800ce120 con 0x7f538c1028b0 2024-03-24T02:58:23.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.552+0000 7f538bfff640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f53740a2800 0x7f53740a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f538c102710 tx=0x7f537c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:23.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.556+0000 7f53897fa640 1 -- 172.21.15.17:0/2071644688 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5380009de0 con 0x7f538c1028b0 2024-03-24T02:58:23.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.783+0000 7f5392510640 1 -- 172.21.15.17:0/2071644688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f538c02d470 con 0x7f538c1028b0 2024-03-24T02:58:23.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.784+0000 7f53897fa640 1 -- 172.21.15.17:0/2071644688 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7658 (secure 0 0 0) 0x7f538009b010 con 0x7f538c1028b0 2024-03-24T02:58:23.783 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:23.783 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-24T02:58:23.783 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 10 2024-03-24T02:58:23.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.786+0000 7f5392510640 1 -- 172.21.15.17:0/2071644688 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f53740a2800 msgr2=0x7f53740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:23.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.786+0000 7f5392510640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f53740a2800 0x7f53740a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f538c102710 tx=0x7f537c005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:23.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.786+0000 7f5392510640 1 -- 172.21.15.17:0/2071644688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f538c1028b0 msgr2=0x7f538c197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.786+0000 7f5392510640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f538c1028b0 0x7f538c197ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f538000d950 tx=0x7f538000de10 comp rx=0 tx=0).stop 2024-03-24T02:58:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.786+0000 7f5392510640 1 -- 172.21.15.17:0/2071644688 shutdown_connections 2024-03-24T02:58:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.786+0000 7f5392510640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f53740a2800 0x7f53740a4cc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.786+0000 7f5392510640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f538c103270 0x7f538c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.786+0000 7f5392510640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f538c1028b0 0x7f538c197ea0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.786+0000 7f5392510640 1 --2- 172.21.15.17:0/2071644688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f538c1016c0 0x7f538c197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.786+0000 7f5392510640 1 -- 172.21.15.17:0/2071644688 >> 172.21.15.17:0/2071644688 conn(0x7f538c0fcc50 msgr2=0x7f538c105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.786+0000 7f5392510640 1 -- 172.21.15.17:0/2071644688 shutdown_connections 2024-03-24T02:58:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:23.786+0000 7f5392510640 1 -- 172.21.15.17:0/2071644688 wait complete. 2024-03-24T02:58:24.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:23 smithi188 ceph-mon[36206]: Deploying daemon mds.g on smithi017 2024-03-24T02:58:24.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:23 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] up:boot 2024-03-24T02:58:24.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:23 smithi188 ceph-mon[36206]: fsmap 9 up:standby 2024-03-24T02:58:24.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:23 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-24T02:58:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:23 smithi019 ceph-mon[35944]: Deploying daemon mds.g on smithi017 2024-03-24T02:58:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:23 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] up:boot 2024-03-24T02:58:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:23 smithi019 ceph-mon[35944]: fsmap 9 up:standby 2024-03-24T02:58:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:23 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-24T02:58:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:23 smithi017 ceph-mon[31926]: Deploying daemon mds.g on smithi017 2024-03-24T02:58:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:23 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] up:boot 2024-03-24T02:58:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:23 smithi017 ceph-mon[31926]: fsmap 9 up:standby 2024-03-24T02:58:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:23 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-24T02:58:24.945 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:24 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2071644688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:24.945 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:24 smithi017 ceph-mon[31926]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:25.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:24 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2071644688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:25.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:24 smithi188 ceph-mon[36206]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:25.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:24 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2071644688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:25.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:24 smithi019 ceph-mon[35944]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:25.456 INFO:journalctl@ceph.mds.g.smithi017.stdout:Mar 24 02:58:25 smithi017 systemd[1]: Started Ceph mds.g for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:58:25.797 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.910+0000 7f80ec762640 1 -- 172.21.15.17:0/1516977303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e40ff990 msgr2=0x7f80e40ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.910+0000 7f80ec762640 1 --2- 172.21.15.17:0/1516977303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e40ff990 0x7f80e40ffdb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009930 tx=0x7f80e002bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.910+0000 7f80ec762640 1 -- 172.21.15.17:0/1516977303 shutdown_connections 2024-03-24T02:58:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.910+0000 7f80ec762640 1 --2- 172.21.15.17:0/1516977303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e4103320 0x7f80e4109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.910+0000 7f80ec762640 1 --2- 172.21.15.17:0/1516977303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80e4102960 0x7f80e4102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.910+0000 7f80ec762640 1 --2- 172.21.15.17:0/1516977303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e40ff990 0x7f80e40ffdb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.910+0000 7f80ec762640 1 -- 172.21.15.17:0/1516977303 >> 172.21.15.17:0/1516977303 conn(0x7f80e40fb280 msgr2=0x7f80e40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.910+0000 7f80ec762640 1 -- 172.21.15.17:0/1516977303 shutdown_connections 2024-03-24T02:58:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.910+0000 7f80ec762640 1 -- 172.21.15.17:0/1516977303 wait complete. 2024-03-24T02:58:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80ec762640 1 Processor -- start 2024-03-24T02:58:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80ec762640 1 -- start start 2024-03-24T02:58:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80ec762640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e40ff990 0x7f80e41022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80ec762640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e4102960 0x7f80e4100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80ec762640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80e4103320 0x7f80e4100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80ec762640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e410d3b0 con 0x7f80e4102960 2024-03-24T02:58:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80ec762640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e410d230 con 0x7f80e4103320 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80ec762640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e410d530 con 0x7f80e40ff990 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80e9cfd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e4102960 0x7f80e4100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80e9cfd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e4102960 0x7f80e4100910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57874/0 (socket says 172.21.15.17:57874) 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80e9cfd640 1 -- 172.21.15.17:0/2662480991 learned_addr learned my addr 172.21.15.17:0/2662480991 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80ea4fe640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e40ff990 0x7f80e41022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80eacff640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80e4103320 0x7f80e4100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80e9cfd640 1 -- 172.21.15.17:0/2662480991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e40ff990 msgr2=0x7f80e41022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80e9cfd640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e40ff990 0x7f80e41022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80e9cfd640 1 -- 172.21.15.17:0/2662480991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80e4103320 msgr2=0x7f80e4100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80e9cfd640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80e4103320 0x7f80e4100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80e9cfd640 1 -- 172.21.15.17:0/2662480991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e41016e0 con 0x7f80e4102960 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80eacff640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80e4103320 0x7f80e4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80ea4fe640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e40ff990 0x7f80e41022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.911+0000 7f80e9cfd640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e4102960 0x7f80e4100910 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f80d400d950 tx=0x7f80d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.912+0000 7f80db7fe640 1 -- 172.21.15.17:0/2662480991 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4017070 con 0x7f80e4102960 2024-03-24T02:58:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.912+0000 7f80db7fe640 1 -- 172.21.15.17:0/2662480991 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80d4009c30 con 0x7f80e4102960 2024-03-24T02:58:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.912+0000 7f80ec762640 1 -- 172.21.15.17:0/2662480991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e41a1b70 con 0x7f80e4102960 2024-03-24T02:58:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.912+0000 7f80db7fe640 1 -- 172.21.15.17:0/2662480991 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d400f6b0 con 0x7f80e4102960 2024-03-24T02:58:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.912+0000 7f80ec762640 1 -- 172.21.15.17:0/2662480991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e41a2110 con 0x7f80e4102960 2024-03-24T02:58:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.913+0000 7f80ec762640 1 -- 172.21.15.17:0/2662480991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e4108ae0 con 0x7f80e4102960 2024-03-24T02:58:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.915+0000 7f80db7fe640 1 -- 172.21.15.17:0/2662480991 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f80d40109d0 con 0x7f80e4102960 2024-03-24T02:58:25.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.915+0000 7f80db7fe640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f80d00a2880 0x7f80d00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:25.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.915+0000 7f80db7fe640 1 -- 172.21.15.17:0/2662480991 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f80d40ce240 con 0x7f80e4102960 2024-03-24T02:58:25.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.915+0000 7f80ea4fe640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f80d00a2880 0x7f80d00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:25.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.916+0000 7f80ea4fe640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f80d00a2880 0x7f80d00a4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009230 tx=0x7f80e002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:25.920+0000 7f80db7fe640 1 -- 172.21.15.17:0/2662480991 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f80d4009de0 con 0x7f80e4102960 2024-03-24T02:58:26.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.147+0000 7f80ec762640 1 -- 172.21.15.17:0/2662480991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80e402d270 con 0x7f80e4102960 2024-03-24T02:58:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.148+0000 7f80db7fe640 1 -- 172.21.15.17:0/2662480991 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8450 (secure 0 0 0) 0x7f80d409b780 con 0x7f80e4102960 2024-03-24T02:58:26.147 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:26.147 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-24T02:58:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 11 2024-03-24T02:58:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.150+0000 7f80ec762640 1 -- 172.21.15.17:0/2662480991 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f80d00a2880 msgr2=0x7f80d00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.150+0000 7f80ec762640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f80d00a2880 0x7f80d00a4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009230 tx=0x7f80e002df90 comp rx=0 tx=0).stop 2024-03-24T02:58:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.150+0000 7f80ec762640 1 -- 172.21.15.17:0/2662480991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e4102960 msgr2=0x7f80e4100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.150+0000 7f80ec762640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e4102960 0x7f80e4100910 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f80d400d950 tx=0x7f80d400de10 comp rx=0 tx=0).stop 2024-03-24T02:58:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.150+0000 7f80ec762640 1 -- 172.21.15.17:0/2662480991 shutdown_connections 2024-03-24T02:58:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.150+0000 7f80ec762640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f80d00a2880 0x7f80d00a4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.150+0000 7f80ec762640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80e4103320 0x7f80e4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.150+0000 7f80ec762640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e4102960 0x7f80e4100910 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.150+0000 7f80ec762640 1 --2- 172.21.15.17:0/2662480991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80e40ff990 0x7f80e41022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.150+0000 7f80ec762640 1 -- 172.21.15.17:0/2662480991 >> 172.21.15.17:0/2662480991 conn(0x7f80e40fb280 msgr2=0x7f80e4105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.150+0000 7f80ec762640 1 -- 172.21.15.17:0/2662480991 shutdown_connections 2024-03-24T02:58:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:26.150+0000 7f80ec762640 1 -- 172.21.15.17:0/2662480991 wait complete. 2024-03-24T02:58:26.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:26 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:26.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:26 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:26.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:26 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:26.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:26 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:26.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:26 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:26.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:26 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:26.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:26 smithi017 ceph-mon[31926]: Deploying daemon mds.a on smithi017 2024-03-24T02:58:26.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:26 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.17:6838/2895019504,v1:172.21.15.17:6839/2895019504] up:boot 2024-03-24T02:58:26.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:26 smithi017 ceph-mon[31926]: fsmap 10 up:standby 2024-03-24T02:58:26.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:26 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-24T02:58:26.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:26 smithi017 ceph-mon[31926]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:26.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:26 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2662480991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:26.613 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:26 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:26.613 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:26 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:26.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:26 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:26.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:26 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:26.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:26 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:26.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:26 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:26.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:26 smithi188 ceph-mon[36206]: Deploying daemon mds.a on smithi017 2024-03-24T02:58:26.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:26 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.17:6838/2895019504,v1:172.21.15.17:6839/2895019504] up:boot 2024-03-24T02:58:26.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:26 smithi188 ceph-mon[36206]: fsmap 10 up:standby 2024-03-24T02:58:26.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:26 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-24T02:58:26.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:26 smithi188 ceph-mon[36206]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:26.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:26 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2662480991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:26.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:26 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:26.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:26 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:26.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:26 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:26.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:26 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:26.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:26 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:26.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:26 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:26.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:26 smithi019 ceph-mon[35944]: Deploying daemon mds.a on smithi017 2024-03-24T02:58:26.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:26 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.17:6838/2895019504,v1:172.21.15.17:6839/2895019504] up:boot 2024-03-24T02:58:26.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:26 smithi019 ceph-mon[35944]: fsmap 10 up:standby 2024-03-24T02:58:26.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:26 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-24T02:58:26.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:26 smithi019 ceph-mon[35944]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:26.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:26 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2662480991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:27.706 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 02:58:27 smithi017 systemd[1]: Starting Ceph mds.a for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T02:58:27.998 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 02:58:27 smithi017 podman[73894]: 2024-03-24 02:58:27.643499542 +0000 UTC m=+0.017736022 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T02:58:27.999 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 02:58:27 smithi017 podman[73894]: 2024-03-24 02:58:27.754819996 +0000 UTC m=+0.129056465 container create 0b11fbe4e1a7539b55bb4ce9f378f17f7ef224783043fd40b2e887af8d040750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T02:58:27.999 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 02:58:27 smithi017 podman[73894]: 2024-03-24 02:58:27.999451235 +0000 UTC m=+0.373687719 container init 0b11fbe4e1a7539b55bb4ce9f378f17f7ef224783043fd40b2e887af8d040750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-24T02:58:28.161 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.268+0000 7f3c555f3640 1 -- 172.21.15.17:0/4107622708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c50104b10 msgr2=0x7f3c50106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.268+0000 7f3c555f3640 1 --2- 172.21.15.17:0/4107622708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c50104b10 0x7f3c50106f00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009900 tx=0x7f3c3802bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.268+0000 7f3c555f3640 1 -- 172.21.15.17:0/4107622708 shutdown_connections 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.268+0000 7f3c555f3640 1 --2- 172.21.15.17:0/4107622708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c50107440 0x7f3c50109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.268+0000 7f3c555f3640 1 --2- 172.21.15.17:0/4107622708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c50104b10 0x7f3c50106f00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.268+0000 7f3c555f3640 1 --2- 172.21.15.17:0/4107622708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c501021e0 0x7f3c501045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.268+0000 7f3c555f3640 1 -- 172.21.15.17:0/4107622708 >> 172.21.15.17:0/4107622708 conn(0x7f3c500fbb30 msgr2=0x7f3c500fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.268+0000 7f3c555f3640 1 -- 172.21.15.17:0/4107622708 shutdown_connections 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.268+0000 7f3c555f3640 1 -- 172.21.15.17:0/4107622708 wait complete. 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.269+0000 7f3c555f3640 1 Processor -- start 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.269+0000 7f3c555f3640 1 -- start start 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.269+0000 7f3c555f3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c501021e0 0x7f3c501979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:28.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.269+0000 7f3c555f3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c50104b10 0x7f3c50197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.269+0000 7f3c555f3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c50107440 0x7f3c5019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.269+0000 7f3c555f3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5010d310 con 0x7f3c50104b10 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.269+0000 7f3c555f3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5010d190 con 0x7f3c501021e0 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.269+0000 7f3c555f3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5010d490 con 0x7f3c50107440 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.269+0000 7f3c467fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c50104b10 0x7f3c50197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.269+0000 7f3c467fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c50104b10 0x7f3c50197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57914/0 (socket says 172.21.15.17:57914) 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.269+0000 7f3c467fc640 1 -- 172.21.15.17:0/333934532 learned_addr learned my addr 172.21.15.17:0/333934532 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.269+0000 7f3c4effd640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c501021e0 0x7f3c501979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c4f7fe640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c50107440 0x7f3c5019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c467fc640 1 -- 172.21.15.17:0/333934532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c50107440 msgr2=0x7f3c5019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c467fc640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c50107440 0x7f3c5019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c467fc640 1 -- 172.21.15.17:0/333934532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c501021e0 msgr2=0x7f3c501979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c467fc640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c501021e0 0x7f3c501979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c467fc640 1 -- 172.21.15.17:0/333934532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5019f600 con 0x7f3c50104b10 2024-03-24T02:58:28.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c4effd640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c501021e0 0x7f3c501979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:28.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c467fc640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c50104b10 0x7f3c50197f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000c00 tx=0x7f3c38005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:28.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c4cff9640 1 -- 172.21.15.17:0/333934532 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38041070 con 0x7f3c50104b10 2024-03-24T02:58:28.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c4cff9640 1 -- 172.21.15.17:0/333934532 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c380044d0 con 0x7f3c50104b10 2024-03-24T02:58:28.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c4cff9640 1 -- 172.21.15.17:0/333934532 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38034440 con 0x7f3c50104b10 2024-03-24T02:58:28.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c555f3640 1 -- 172.21.15.17:0/333934532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5019f890 con 0x7f3c50104b10 2024-03-24T02:58:28.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.270+0000 7f3c555f3640 1 -- 172.21.15.17:0/333934532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5019fd30 con 0x7f3c50104b10 2024-03-24T02:58:28.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.271+0000 7f3c555f3640 1 -- 172.21.15.17:0/333934532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c500fd620 con 0x7f3c50104b10 2024-03-24T02:58:28.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.272+0000 7f3c4cff9640 1 -- 172.21.15.17:0/333934532 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3c38044070 con 0x7f3c50104b10 2024-03-24T02:58:28.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.273+0000 7f3c4cff9640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3c3c0a2880 0x7f3c3c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:28.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.273+0000 7f3c4effd640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3c3c0a2880 0x7f3c3c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:28.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.273+0000 7f3c4cff9640 1 -- 172.21.15.17:0/333934532 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3c380f0380 con 0x7f3c50104b10 2024-03-24T02:58:28.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.273+0000 7f3c4effd640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3c3c0a2880 0x7f3c3c0a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3c400058c0 tx=0x7f3c40005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:28.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.276+0000 7f3c4cff9640 1 -- 172.21.15.17:0/333934532 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3c380bd8c0 con 0x7f3c50104b10 2024-03-24T02:58:28.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:28 smithi188 ceph-mon[36206]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:28.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:28 smithi019 ceph-mon[35944]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:28.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:28 smithi017 ceph-mon[31926]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:28.456 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 02:58:28 smithi017 podman[73894]: 2024-03-24 02:58:28.002352955 +0000 UTC m=+0.376589425 container start 0b11fbe4e1a7539b55bb4ce9f378f17f7ef224783043fd40b2e887af8d040750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-a, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-24T02:58:28.456 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 02:58:28 smithi017 ceph-mds[73913]: -- 172.21.15.17:0/4135550111 <== mon.0 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a0aa76ba40 con 0x55a0aa73e800 2024-03-24T02:58:28.456 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 02:58:28 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-a[73909]: starting mds.a at 2024-03-24T02:58:28.456 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 02:58:28 smithi017 bash[73894]: 0b11fbe4e1a7539b55bb4ce9f378f17f7ef224783043fd40b2e887af8d040750 2024-03-24T02:58:28.456 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 02:58:28 smithi017 systemd[1]: Started Ceph mds.a for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:58:28.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.500+0000 7f3c555f3640 1 -- 172.21.15.17:0/333934532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c5002d230 con 0x7f3c50104b10 2024-03-24T02:58:28.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.501+0000 7f3c4cff9640 1 -- 172.21.15.17:0/333934532 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8450 (secure 0 0 0) 0x7f3c380bd1c0 con 0x7f3c50104b10 2024-03-24T02:58:28.500 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:28.501 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-24T02:58:28.501 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 11 2024-03-24T02:58:28.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.503+0000 7f3c555f3640 1 -- 172.21.15.17:0/333934532 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3c3c0a2880 msgr2=0x7f3c3c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.503+0000 7f3c555f3640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3c3c0a2880 0x7f3c3c0a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3c400058c0 tx=0x7f3c40005850 comp rx=0 tx=0).stop 2024-03-24T02:58:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.503+0000 7f3c555f3640 1 -- 172.21.15.17:0/333934532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c50104b10 msgr2=0x7f3c50197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.503+0000 7f3c555f3640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c50104b10 0x7f3c50197f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000c00 tx=0x7f3c38005d50 comp rx=0 tx=0).stop 2024-03-24T02:58:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.503+0000 7f3c555f3640 1 -- 172.21.15.17:0/333934532 shutdown_connections 2024-03-24T02:58:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.503+0000 7f3c555f3640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3c3c0a2880 0x7f3c3c0a4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.503+0000 7f3c555f3640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c50107440 0x7f3c5019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.503+0000 7f3c555f3640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c50104b10 0x7f3c50197f20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.503+0000 7f3c555f3640 1 --2- 172.21.15.17:0/333934532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c501021e0 0x7f3c501979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.503+0000 7f3c555f3640 1 -- 172.21.15.17:0/333934532 >> 172.21.15.17:0/333934532 conn(0x7f3c500fbb30 msgr2=0x7f3c50102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.503+0000 7f3c555f3640 1 -- 172.21.15.17:0/333934532 shutdown_connections 2024-03-24T02:58:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:28.503+0000 7f3c555f3640 1 -- 172.21.15.17:0/333934532 wait complete. 2024-03-24T02:58:29.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:29.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:29.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:29.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:29.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:29.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:29.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:29 smithi019 ceph-mon[35944]: Deploying daemon mds.b on smithi019 2024-03-24T02:58:29.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:29 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/333934532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:29.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:29 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.17:6840/1592961709,v1:172.21.15.17:6841/1592961709] up:boot 2024-03-24T02:58:29.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:29 smithi019 ceph-mon[35944]: fsmap 11 up:standby 2024-03-24T02:58:29.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-24T02:58:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:29 smithi017 ceph-mon[31926]: Deploying daemon mds.b on smithi019 2024-03-24T02:58:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:29 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/333934532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:29 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.17:6840/1592961709,v1:172.21.15.17:6841/1592961709] up:boot 2024-03-24T02:58:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:29 smithi017 ceph-mon[31926]: fsmap 11 up:standby 2024-03-24T02:58:29.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-24T02:58:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T02:58:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T02:58:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:29 smithi188 ceph-mon[36206]: Deploying daemon mds.b on smithi019 2024-03-24T02:58:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:29 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/333934532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:29 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.17:6840/1592961709,v1:172.21.15.17:6841/1592961709] up:boot 2024-03-24T02:58:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:29 smithi188 ceph-mon[36206]: fsmap 11 up:standby 2024-03-24T02:58:29.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-24T02:58:30.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:30 smithi017 ceph-mon[31926]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:30.457 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:30 smithi019 ceph-mon[35944]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:30.515 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:30.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:30 smithi188 ceph-mon[36206]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:30.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.630+0000 7f6e03885640 1 -- 172.21.15.17:0/3386312358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc1017d0 msgr2=0x7f6dfc101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:30.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.630+0000 7f6e03885640 1 --2- 172.21.15.17:0/3386312358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc1017d0 0x7f6dfc101c50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009900 tx=0x7f6dec02bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:30.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 -- 172.21.15.17:0/3386312358 shutdown_connections 2024-03-24T02:58:30.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 --2- 172.21.15.17:0/3386312358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dfc102190 0x7f6dfc108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:30.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 --2- 172.21.15.17:0/3386312358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc1017d0 0x7f6dfc101c50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:30.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 --2- 172.21.15.17:0/3386312358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dfc1005e0 0x7f6dfc100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:30.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 -- 172.21.15.17:0/3386312358 >> 172.21.15.17:0/3386312358 conn(0x7f6dfc0fbb50 msgr2=0x7f6dfc0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:30.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 -- 172.21.15.17:0/3386312358 shutdown_connections 2024-03-24T02:58:30.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 -- 172.21.15.17:0/3386312358 wait complete. 2024-03-24T02:58:30.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 Processor -- start 2024-03-24T02:58:30.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 -- start start 2024-03-24T02:58:30.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dfc1005e0 0x7f6dfc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:30.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dfc1017d0 0x7f6dfc19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:30.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc102190 0x7f6dfc19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:30.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc10c260 con 0x7f6dfc102190 2024-03-24T02:58:30.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc10c0e0 con 0x7f6dfc1005e0 2024-03-24T02:58:30.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.631+0000 7f6e03885640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc10c3e0 con 0x7f6dfc1017d0 2024-03-24T02:58:30.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e01e22640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc102190 0x7f6dfc19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e01e22640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc102190 0x7f6dfc19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58054/0 (socket says 172.21.15.17:58054) 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e01e22640 1 -- 172.21.15.17:0/2212714431 learned_addr learned my addr 172.21.15.17:0/2212714431 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e01621640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dfc1005e0 0x7f6dfc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e00e20640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dfc1017d0 0x7f6dfc19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e01e22640 1 -- 172.21.15.17:0/2212714431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dfc1017d0 msgr2=0x7f6dfc19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e01e22640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dfc1017d0 0x7f6dfc19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e01e22640 1 -- 172.21.15.17:0/2212714431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dfc1005e0 msgr2=0x7f6dfc06a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e01e22640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dfc1005e0 0x7f6dfc06a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e01e22640 1 -- 172.21.15.17:0/2212714431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc19f620 con 0x7f6dfc102190 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e01e22640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc102190 0x7f6dfc19ed10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6df400eaa0 tx=0x7f6df400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6df27fc640 1 -- 172.21.15.17:0/2212714431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df400cdb0 con 0x7f6dfc102190 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e03885640 1 -- 172.21.15.17:0/2212714431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc19f8b0 con 0x7f6dfc102190 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6df27fc640 1 -- 172.21.15.17:0/2212714431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6df4005020 con 0x7f6dfc102190 2024-03-24T02:58:30.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6df27fc640 1 -- 172.21.15.17:0/2212714431 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df40116d0 con 0x7f6dfc102190 2024-03-24T02:58:30.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.632+0000 7f6e03885640 1 -- 172.21.15.17:0/2212714431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc19fdd0 con 0x7f6dfc102190 2024-03-24T02:58:30.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.633+0000 7f6e03885640 1 -- 172.21.15.17:0/2212714431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc107900 con 0x7f6dfc102190 2024-03-24T02:58:30.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.634+0000 7f6df27fc640 1 -- 172.21.15.17:0/2212714431 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6df40054d0 con 0x7f6dfc102190 2024-03-24T02:58:30.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.635+0000 7f6df27fc640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6de80a2800 0x7f6de80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:30.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.635+0000 7f6e01621640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6de80a2800 0x7f6de80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:30.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.635+0000 7f6df27fc640 1 -- 172.21.15.17:0/2212714431 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6df40cec40 con 0x7f6dfc102190 2024-03-24T02:58:30.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.636+0000 7f6e01621640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6de80a2800 0x7f6de80a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6df8005fd0 tx=0x7f6df8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:30.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.639+0000 7f6df27fc640 1 -- 172.21.15.17:0/2212714431 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6df4016030 con 0x7f6dfc102190 2024-03-24T02:58:30.708 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 02:58:30 smithi019 podman[66477]: 2024-03-24 02:58:30.581955664 +0000 UTC m=+0.018203284 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T02:58:30.709 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 02:58:30 smithi019 podman[66477]: 2024-03-24 02:58:30.709893944 +0000 UTC m=+0.146141584 container create 0c7ee800196e6ff2fdfba445d337b3f7d4c03f7e5926ce9a89641b4ed4af4c79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-24T02:58:30.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.861+0000 7f6e03885640 1 -- 172.21.15.17:0/2212714431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dfc02d070 con 0x7f6dfc102190 2024-03-24T02:58:30.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.861+0000 7f6df27fc640 1 -- 172.21.15.17:0/2212714431 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9242 (secure 0 0 0) 0x7f6df409c180 con 0x7f6dfc102190 2024-03-24T02:58:30.861 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:30.861 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/1592961709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":1592961709},{"type":"v1","addr":"172.21.15.17:6841","nonce":1592961709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-24T02:58:30.861 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 12 2024-03-24T02:58:30.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.863+0000 7f6e03885640 1 -- 172.21.15.17:0/2212714431 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6de80a2800 msgr2=0x7f6de80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:30.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.863+0000 7f6e03885640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6de80a2800 0x7f6de80a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6df8005fd0 tx=0x7f6df8005850 comp rx=0 tx=0).stop 2024-03-24T02:58:30.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.863+0000 7f6e03885640 1 -- 172.21.15.17:0/2212714431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc102190 msgr2=0x7f6dfc19ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:30.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.863+0000 7f6e03885640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc102190 0x7f6dfc19ed10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6df400eaa0 tx=0x7f6df400ef60 comp rx=0 tx=0).stop 2024-03-24T02:58:30.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.863+0000 7f6e03885640 1 -- 172.21.15.17:0/2212714431 shutdown_connections 2024-03-24T02:58:30.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.863+0000 7f6e03885640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6de80a2800 0x7f6de80a4cc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:30.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.863+0000 7f6e03885640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dfc102190 0x7f6dfc19ed10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:30.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.863+0000 7f6e03885640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dfc1017d0 0x7f6dfc19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:30.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.863+0000 7f6e03885640 1 --2- 172.21.15.17:0/2212714431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dfc1005e0 0x7f6dfc06a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:30.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.863+0000 7f6e03885640 1 -- 172.21.15.17:0/2212714431 >> 172.21.15.17:0/2212714431 conn(0x7f6dfc0fbb50 msgr2=0x7f6dfc104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:30.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.864+0000 7f6e03885640 1 -- 172.21.15.17:0/2212714431 shutdown_connections 2024-03-24T02:58:30.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:30.864+0000 7f6e03885640 1 -- 172.21.15.17:0/2212714431 wait complete. 2024-03-24T02:58:30.978 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 02:58:30 smithi019 podman[66477]: 2024-03-24 02:58:30.954208332 +0000 UTC m=+0.390455958 container init 0c7ee800196e6ff2fdfba445d337b3f7d4c03f7e5926ce9a89641b4ed4af4c79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-03-24T02:58:30.979 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 02:58:30 smithi019 podman[66477]: 2024-03-24 02:58:30.95680276 +0000 UTC m=+0.393050360 container start 0c7ee800196e6ff2fdfba445d337b3f7d4c03f7e5926ce9a89641b4ed4af4c79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T02:58:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:31 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2212714431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:31.321 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 02:58:30 smithi019 ceph-mds[66496]: -- 172.21.15.19:0/630292859 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5614fb68fa40 con 0x5614fb662800 2024-03-24T02:58:31.321 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 02:58:30 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b[66492]: starting mds.b at 2024-03-24T02:58:31.321 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 02:58:31 smithi019 bash[66477]: 0c7ee800196e6ff2fdfba445d337b3f7d4c03f7e5926ce9a89641b4ed4af4c79 2024-03-24T02:58:31.321 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 02:58:31 smithi019 systemd[1]: Started Ceph mds.b for 2a682800-e989-11ee-95cd-87774f69a715. 2024-03-24T02:58:31.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:31 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2212714431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:31.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:31.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:31.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:31.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:31 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2212714431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:31.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:31.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:31.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:32.420 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:32.420 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:32 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] up:boot 2024-03-24T02:58:32.420 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:32 smithi017 ceph-mon[31926]: fsmap 12 up:standby 2024-03-24T02:58:32.420 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-24T02:58:32.420 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:32.420 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:32 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:58:32.420 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:32 smithi017 ceph-mon[31926]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:32.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:32.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:32 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] up:boot 2024-03-24T02:58:32.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:32 smithi188 ceph-mon[36206]: fsmap 12 up:standby 2024-03-24T02:58:32.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-24T02:58:32.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:32.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:32 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:58:32.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:32 smithi188 ceph-mon[36206]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:32.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:32.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:32 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] up:boot 2024-03-24T02:58:32.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:32 smithi019 ceph-mon[35944]: fsmap 12 up:standby 2024-03-24T02:58:32.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-24T02:58:32.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:32.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:32 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:58:32.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:32 smithi019 ceph-mon[35944]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:32.874 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:32.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.988+0000 7fcdc8e36640 1 -- 172.21.15.17:0/1916138494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc41016b0 msgr2=0x7fcdc4101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:32.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.988+0000 7fcdc8e36640 1 --2- 172.21.15.17:0/1916138494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc41016b0 0x7fcdc4101ad0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009900 tx=0x7fcdb402bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:32.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.988+0000 7fcdc8e36640 1 -- 172.21.15.17:0/1916138494 shutdown_connections 2024-03-24T02:58:32.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.988+0000 7fcdc8e36640 1 --2- 172.21.15.17:0/1916138494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4103260 0x7fcdc4109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:32.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.988+0000 7fcdc8e36640 1 --2- 172.21.15.17:0/1916138494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc41028a0 0x7fcdc4102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:32.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.988+0000 7fcdc8e36640 1 --2- 172.21.15.17:0/1916138494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc41016b0 0x7fcdc4101ad0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:32.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.988+0000 7fcdc8e36640 1 -- 172.21.15.17:0/1916138494 >> 172.21.15.17:0/1916138494 conn(0x7fcdc40fcc80 msgr2=0x7fcdc40ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:32.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.988+0000 7fcdc8e36640 1 -- 172.21.15.17:0/1916138494 shutdown_connections 2024-03-24T02:58:32.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.988+0000 7fcdc8e36640 1 -- 172.21.15.17:0/1916138494 wait complete. 2024-03-24T02:58:32.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.989+0000 7fcdc8e36640 1 Processor -- start 2024-03-24T02:58:32.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.989+0000 7fcdc8e36640 1 -- start start 2024-03-24T02:58:32.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.989+0000 7fcdc8e36640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc41016b0 0x7fcdc4197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:32.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.989+0000 7fcdc8e36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc41028a0 0x7fcdc4197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:32.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc8e36640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc4103260 0x7fcdc419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:32.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc8e36640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc410d300 con 0x7fcdc41016b0 2024-03-24T02:58:32.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc8e36640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc410d180 con 0x7fcdc41028a0 2024-03-24T02:58:32.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc8e36640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc410d480 con 0x7fcdc4103260 2024-03-24T02:58:32.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc41016b0 0x7fcdc4197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc41016b0 0x7fcdc4197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58072/0 (socket says 172.21.15.17:58072) 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc259c640 1 -- 172.21.15.17:0/468417140 learned_addr learned my addr 172.21.15.17:0/468417140 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdbbfff640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc41028a0 0x7fcdc4197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc2d9d640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc4103260 0x7fcdc419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc259c640 1 -- 172.21.15.17:0/468417140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc4103260 msgr2=0x7fcdc419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc259c640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc4103260 0x7fcdc419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc259c640 1 -- 172.21.15.17:0/468417140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc41028a0 msgr2=0x7fcdc4197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc259c640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc41028a0 0x7fcdc4197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc259c640 1 -- 172.21.15.17:0/468417140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc419f570 con 0x7fcdc41016b0 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc259c640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc41016b0 0x7fcdc4197950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4000c00 tx=0x7fcdb4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/468417140 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4041070 con 0x7fcdc41016b0 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/468417140 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdb40044d0 con 0x7fcdc41016b0 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc8e36640 1 -- 172.21.15.17:0/468417140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc419f800 con 0x7fcdc41016b0 2024-03-24T02:58:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/468417140 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4034440 con 0x7fcdc41016b0 2024-03-24T02:58:32.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.990+0000 7fcdc8e36640 1 -- 172.21.15.17:0/468417140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc419fcf0 con 0x7fcdc41016b0 2024-03-24T02:58:32.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.992+0000 7fcdc8e36640 1 -- 172.21.15.17:0/468417140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc41089d0 con 0x7fcdc41016b0 2024-03-24T02:58:32.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.993+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/468417140 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fcdb4044070 con 0x7fcdc41016b0 2024-03-24T02:58:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.993+0000 7fcdbb7fe640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcdb00a2800 0x7fcdb00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.993+0000 7fcdbbfff640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcdb00a2800 0x7fcdb00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.993+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/468417140 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcdb40f0130 con 0x7fcdc41016b0 2024-03-24T02:58:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.994+0000 7fcdbbfff640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcdb00a2800 0x7fcdb00a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4198e60 tx=0x7fcdac00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:32.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:32.998+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/468417140 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcdb40bd5d0 con 0x7fcdc41016b0 2024-03-24T02:58:33.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.246+0000 7fcdc8e36640 1 -- 172.21.15.17:0/468417140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdc40622a0 con 0x7fcdc41016b0 2024-03-24T02:58:33.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.246+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/468417140 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10034 (secure 0 0 0) 0x7fcdb40bced0 con 0x7fcdc41016b0 2024-03-24T02:58:33.246 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:33.246 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/1592961709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":1592961709},{"type":"v1","addr":"172.21.15.17:6841","nonce":1592961709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2102503456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2102503456},{"type":"v1","addr":"172.21.15.19:6839","nonce":2102503456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-24T02:58:33.246 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 13 2024-03-24T02:58:33.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.248+0000 7fcdc8e36640 1 -- 172.21.15.17:0/468417140 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcdb00a2800 msgr2=0x7fcdb00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:33.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.248+0000 7fcdc8e36640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcdb00a2800 0x7fcdb00a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4198e60 tx=0x7fcdac00a2d0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.248+0000 7fcdc8e36640 1 -- 172.21.15.17:0/468417140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc41016b0 msgr2=0x7fcdc4197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:33.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.248+0000 7fcdc8e36640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc41016b0 0x7fcdc4197950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4000c00 tx=0x7fcdb4005d50 comp rx=0 tx=0).stop 2024-03-24T02:58:33.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.249+0000 7fcdc8e36640 1 -- 172.21.15.17:0/468417140 shutdown_connections 2024-03-24T02:58:33.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.249+0000 7fcdc8e36640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcdb00a2800 0x7fcdb00a4cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.249+0000 7fcdc8e36640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc4103260 0x7fcdc419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.249+0000 7fcdc8e36640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc41028a0 0x7fcdc4197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.249+0000 7fcdc8e36640 1 --2- 172.21.15.17:0/468417140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc41016b0 0x7fcdc4197950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.249+0000 7fcdc8e36640 1 -- 172.21.15.17:0/468417140 >> 172.21.15.17:0/468417140 conn(0x7fcdc40fcc80 msgr2=0x7fcdc4105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:33.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.249+0000 7fcdc8e36640 1 -- 172.21.15.17:0/468417140 shutdown_connections 2024-03-24T02:58:33.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.249+0000 7fcdc8e36640 1 -- 172.21.15.17:0/468417140 wait complete. 2024-03-24T02:58:33.259 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T02:58:33.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.363+0000 7f0577fa9640 1 -- 172.21.15.17:0/1018885923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570102190 msgr2=0x7f0570108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:33.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 --2- 172.21.15.17:0/1018885923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570102190 0x7f0570108a20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0568009ca0 tx=0x7f056802c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 -- 172.21.15.17:0/1018885923 shutdown_connections 2024-03-24T02:58:33.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 --2- 172.21.15.17:0/1018885923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570102190 0x7f0570108a20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 --2- 172.21.15.17:0/1018885923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05701017d0 0x7f0570101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 --2- 172.21.15.17:0/1018885923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05701005e0 0x7f0570100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 -- 172.21.15.17:0/1018885923 >> 172.21.15.17:0/1018885923 conn(0x7f05700fbb50 msgr2=0x7f05700fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:33.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 -- 172.21.15.17:0/1018885923 shutdown_connections 2024-03-24T02:58:33.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 -- 172.21.15.17:0/1018885923 wait complete. 2024-03-24T02:58:33.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 Processor -- start 2024-03-24T02:58:33.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 -- start start 2024-03-24T02:58:33.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05701005e0 0x7f0570197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:33.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05701017d0 0x7f0570197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:33.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0570102190 0x7f057019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:33.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057010c260 con 0x7f05701005e0 2024-03-24T02:58:33.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057010c0e0 con 0x7f05701017d0 2024-03-24T02:58:33.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.364+0000 7f0577fa9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057010c3e0 con 0x7f0570102190 2024-03-24T02:58:33.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0575d45640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05701005e0 0x7f0570197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:33.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0575d45640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05701005e0 0x7f0570197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58102/0 (socket says 172.21.15.17:58102) 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0575d45640 1 -- 172.21.15.17:0/1277252165 learned_addr learned my addr 172.21.15.17:0/1277252165 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0576546640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0570102190 0x7f057019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0575544640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05701017d0 0x7f0570197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0575d45640 1 -- 172.21.15.17:0/1277252165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0570102190 msgr2=0x7f057019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0575d45640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0570102190 0x7f057019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0575d45640 1 -- 172.21.15.17:0/1277252165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05701017d0 msgr2=0x7f0570197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0575d45640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05701017d0 0x7f0570197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0575d45640 1 -- 172.21.15.17:0/1277252165 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057019f630 con 0x7f05701005e0 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0575d45640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05701005e0 0x7f0570197a10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f056c00e9c0 tx=0x7f056c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0566ffd640 1 -- 172.21.15.17:0/1277252165 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c00cd80 con 0x7f05701005e0 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0577fa9640 1 -- 172.21.15.17:0/1277252165 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057019f920 con 0x7f05701005e0 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0566ffd640 1 -- 172.21.15.17:0/1277252165 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f056c0035f0 con 0x7f05701005e0 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0566ffd640 1 -- 172.21.15.17:0/1277252165 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c010690 con 0x7f05701005e0 2024-03-24T02:58:33.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.365+0000 7f0577fa9640 1 -- 172.21.15.17:0/1277252165 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057019fec0 con 0x7f05701005e0 2024-03-24T02:58:33.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.366+0000 7f0577fa9640 1 -- 172.21.15.17:0/1277252165 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05701a0150 con 0x7f05701005e0 2024-03-24T02:58:33.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.367+0000 7f0566ffd640 1 -- 172.21.15.17:0/1277252165 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f056c0037a0 con 0x7f05701005e0 2024-03-24T02:58:33.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.368+0000 7f0566ffd640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f055c0a2800 0x7f055c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:33.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.368+0000 7f0575544640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f055c0a2800 0x7f055c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:33.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.368+0000 7f0566ffd640 1 -- 172.21.15.17:0/1277252165 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f056c0cea20 con 0x7f05701005e0 2024-03-24T02:58:33.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.368+0000 7f0575544640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f055c0a2800 0x7f055c0a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0570198f20 tx=0x7f0560005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:33.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.376+0000 7f0566ffd640 1 -- 172.21.15.17:0/1277252165 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f056c09bf60 con 0x7f05701005e0 2024-03-24T02:58:33.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.562+0000 7f0577fa9640 1 -- 172.21.15.17:0/1277252165 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05701a0320 con 0x7f05701005e0 2024-03-24T02:58:33.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.563+0000 7f0566ffd640 1 -- 172.21.15.17:0/1277252165 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18428 (secure 0 0 0) 0x7f056c09b860 con 0x7f05701005e0 2024-03-24T02:58:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:33.562 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":81,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:57:45.021539+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:58:33.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.565+0000 7f0577fa9640 1 -- 172.21.15.17:0/1277252165 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f055c0a2800 msgr2=0x7f055c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:33.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.565+0000 7f0577fa9640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f055c0a2800 0x7f055c0a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0570198f20 tx=0x7f0560005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:33.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.565+0000 7f0577fa9640 1 -- 172.21.15.17:0/1277252165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05701005e0 msgr2=0x7f0570197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:33.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.565+0000 7f0577fa9640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05701005e0 0x7f0570197a10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f056c00e9c0 tx=0x7f056c00ee80 comp rx=0 tx=0).stop 2024-03-24T02:58:33.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.565+0000 7f0577fa9640 1 -- 172.21.15.17:0/1277252165 shutdown_connections 2024-03-24T02:58:33.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.565+0000 7f0577fa9640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f055c0a2800 0x7f055c0a4cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.565+0000 7f0577fa9640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0570102190 0x7f057019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.565+0000 7f0577fa9640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05701017d0 0x7f0570197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.565+0000 7f0577fa9640 1 --2- 172.21.15.17:0/1277252165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05701005e0 0x7f0570197a10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.565+0000 7f0577fa9640 1 -- 172.21.15.17:0/1277252165 >> 172.21.15.17:0/1277252165 conn(0x7f05700fbb50 msgr2=0x7f05700fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:33.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.565+0000 7f0577fa9640 1 -- 172.21.15.17:0/1277252165 shutdown_connections 2024-03-24T02:58:33.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.565+0000 7f0577fa9640 1 -- 172.21.15.17:0/1277252165 wait complete. 2024-03-24T02:58:33.575 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T02:53:54.207139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-24T02:58:33.576 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T02:58:33.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.683+0000 7f2dc7122640 1 -- 172.21.15.17:0/1188138378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dc0101790 msgr2=0x7f2dc0101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:33.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.683+0000 7f2dc7122640 1 --2- 172.21.15.17:0/1188138378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dc0101790 0x7f2dc0101c10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009900 tx=0x7f2dac02bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:33.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.683+0000 7f2dc7122640 1 -- 172.21.15.17:0/1188138378 shutdown_connections 2024-03-24T02:58:33.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.683+0000 7f2dc7122640 1 --2- 172.21.15.17:0/1188138378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc0102150 0x7f2dc01089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.683+0000 7f2dc7122640 1 --2- 172.21.15.17:0/1188138378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dc0101790 0x7f2dc0101c10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.683+0000 7f2dc7122640 1 --2- 172.21.15.17:0/1188138378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc01005a0 0x7f2dc01009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.683+0000 7f2dc7122640 1 -- 172.21.15.17:0/1188138378 >> 172.21.15.17:0/1188138378 conn(0x7f2dc00fbb30 msgr2=0x7f2dc00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:33.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.683+0000 7f2dc7122640 1 -- 172.21.15.17:0/1188138378 shutdown_connections 2024-03-24T02:58:33.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.683+0000 7f2dc7122640 1 -- 172.21.15.17:0/1188138378 wait complete. 2024-03-24T02:58:33.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc7122640 1 Processor -- start 2024-03-24T02:58:33.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc7122640 1 -- start start 2024-03-24T02:58:33.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc7122640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dc01005a0 0x7f2dc0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:33.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc7122640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc0101790 0x7f2dc0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:33.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc7122640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc0102150 0x7f2dc019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:33.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc7122640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc010c230 con 0x7f2dc01005a0 2024-03-24T02:58:33.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc7122640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc010c0b0 con 0x7f2dc0102150 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc7122640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc010c3b0 con 0x7f2dc0101790 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc4ebe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dc01005a0 0x7f2dc0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc4ebe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dc01005a0 0x7f2dc0197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58118/0 (socket says 172.21.15.17:58118) 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc4ebe640 1 -- 172.21.15.17:0/2979294787 learned_addr learned my addr 172.21.15.17:0/2979294787 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc56bf640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc0102150 0x7f2dc019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dbffff640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc0101790 0x7f2dc0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc4ebe640 1 -- 172.21.15.17:0/2979294787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc0101790 msgr2=0x7f2dc0197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc4ebe640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc0101790 0x7f2dc0197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc4ebe640 1 -- 172.21.15.17:0/2979294787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc0102150 msgr2=0x7f2dc019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc4ebe640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc0102150 0x7f2dc019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc4ebe640 1 -- 172.21.15.17:0/2979294787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc019f5e0 con 0x7f2dc01005a0 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.684+0000 7f2dc56bf640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc0102150 0x7f2dc019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.685+0000 7f2dc4ebe640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dc01005a0 0x7f2dc0197a10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2db400d950 tx=0x7f2db400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.685+0000 7f2dbdffb640 1 -- 172.21.15.17:0/2979294787 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4017020 con 0x7f2dc01005a0 2024-03-24T02:58:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.685+0000 7f2dbdffb640 1 -- 172.21.15.17:0/2979294787 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2db4009c30 con 0x7f2dc01005a0 2024-03-24T02:58:33.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.685+0000 7f2dc7122640 1 -- 172.21.15.17:0/2979294787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc019f8d0 con 0x7f2dc01005a0 2024-03-24T02:58:33.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.685+0000 7f2dbdffb640 1 -- 172.21.15.17:0/2979294787 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db400f720 con 0x7f2dc01005a0 2024-03-24T02:58:33.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.685+0000 7f2dc7122640 1 -- 172.21.15.17:0/2979294787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc019fdf0 con 0x7f2dc01005a0 2024-03-24T02:58:33.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.686+0000 7f2dc7122640 1 -- 172.21.15.17:0/2979294787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc01a0050 con 0x7f2dc01005a0 2024-03-24T02:58:33.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.687+0000 7f2dbdffb640 1 -- 172.21.15.17:0/2979294787 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2db401a020 con 0x7f2dc01005a0 2024-03-24T02:58:33.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.687+0000 7f2dbdffb640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2db00a2800 0x7f2db00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:33.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.688+0000 7f2dbffff640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2db00a2800 0x7f2db00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:33.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.688+0000 7f2dbdffb640 1 -- 172.21.15.17:0/2979294787 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2db40ce4d0 con 0x7f2dc01005a0 2024-03-24T02:58:33.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.688+0000 7f2dbffff640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2db00a2800 0x7f2db00a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2dac02c7b0 tx=0x7f2dac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:33.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.692+0000 7f2dbdffb640 1 -- 172.21.15.17:0/2979294787 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2db4021020 con 0x7f2dc01005a0 2024-03-24T02:58:33.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.891+0000 7f2dc7122640 1 -- 172.21.15.17:0/2979294787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2dc01a0220 con 0x7f2dc01005a0 2024-03-24T02:58:33.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.892+0000 7f2dbdffb640 1 -- 172.21.15.17:0/2979294787 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2db4009de0 con 0x7f2dc01005a0 2024-03-24T02:58:33.891 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-24T02:58:33.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.894+0000 7f2dc7122640 1 -- 172.21.15.17:0/2979294787 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2db00a2800 msgr2=0x7f2db00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:33.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.894+0000 7f2dc7122640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2db00a2800 0x7f2db00a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2dac02c7b0 tx=0x7f2dac005fd0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.894+0000 7f2dc7122640 1 -- 172.21.15.17:0/2979294787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dc01005a0 msgr2=0x7f2dc0197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:33.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.894+0000 7f2dc7122640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dc01005a0 0x7f2dc0197a10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2db400d950 tx=0x7f2db400de10 comp rx=0 tx=0).stop 2024-03-24T02:58:33.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.894+0000 7f2dc7122640 1 -- 172.21.15.17:0/2979294787 shutdown_connections 2024-03-24T02:58:33.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.894+0000 7f2dc7122640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2db00a2800 0x7f2db00a4cc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.894+0000 7f2dc7122640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc0102150 0x7f2dc019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.894+0000 7f2dc7122640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc0101790 0x7f2dc0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.894+0000 7f2dc7122640 1 --2- 172.21.15.17:0/2979294787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2dc01005a0 0x7f2dc0197a10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:33.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.895+0000 7f2dc7122640 1 -- 172.21.15.17:0/2979294787 >> 172.21.15.17:0/2979294787 conn(0x7f2dc00fbb30 msgr2=0x7f2dc00fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:33.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.895+0000 7f2dc7122640 1 -- 172.21.15.17:0/2979294787 shutdown_connections 2024-03-24T02:58:33.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:33.895+0000 7f2dc7122640 1 -- 172.21.15.17:0/2979294787 wait complete. 2024-03-24T02:58:33.905 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-24T02:58:34.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.010+0000 7f719bd7e640 1 -- 172.21.15.17:0/212090933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7194104ad0 msgr2=0x7f7194106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:34.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.010+0000 7f719bd7e640 1 --2- 172.21.15.17:0/212090933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7194104ad0 0x7f7194106ec0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7188009900 tx=0x7f718802bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:34.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 -- 172.21.15.17:0/212090933 shutdown_connections 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 --2- 172.21.15.17:0/212090933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7194107400 0x7f71941097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 --2- 172.21.15.17:0/212090933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7194104ad0 0x7f7194106ec0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 --2- 172.21.15.17:0/212090933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71941021a0 0x7f7194104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 -- 172.21.15.17:0/212090933 >> 172.21.15.17:0/212090933 conn(0x7f71940fbb50 msgr2=0x7f71940fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 -- 172.21.15.17:0/212090933 shutdown_connections 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 -- 172.21.15.17:0/212090933 wait complete. 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 Processor -- start 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 -- start start 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71941021a0 0x7f7194197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7194104ad0 0x7f7194197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7194107400 0x7f719419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719410d360 con 0x7f71941021a0 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719410d1e0 con 0x7f7194104ad0 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f719bd7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719410d4e0 con 0x7f7194107400 2024-03-24T02:58:34.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f7199b1a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71941021a0 0x7f7194197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f7199b1a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71941021a0 0x7f7194197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58144/0 (socket says 172.21.15.17:58144) 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.011+0000 7f7199b1a640 1 -- 172.21.15.17:0/2117598403 learned_addr learned my addr 172.21.15.17:0/2117598403 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f7199319640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7194104ad0 0x7f7194197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f719a31b640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7194107400 0x7f719419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f7199b1a640 1 -- 172.21.15.17:0/2117598403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7194107400 msgr2=0x7f719419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f7199b1a640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7194107400 0x7f719419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f7199b1a640 1 -- 172.21.15.17:0/2117598403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7194104ad0 msgr2=0x7f7194197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f7199b1a640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7194104ad0 0x7f7194197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f7199b1a640 1 -- 172.21.15.17:0/2117598403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719419f630 con 0x7f71941021a0 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f7199b1a640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71941021a0 0x7f7194197a10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f718400ebc0 tx=0x7f718400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f7192ffd640 1 -- 172.21.15.17:0/2117598403 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184010630 con 0x7f71941021a0 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f7192ffd640 1 -- 172.21.15.17:0/2117598403 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7184010cb0 con 0x7f71941021a0 2024-03-24T02:58:34.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f719bd7e640 1 -- 172.21.15.17:0/2117598403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719419f920 con 0x7f71941021a0 2024-03-24T02:58:34.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f7192ffd640 1 -- 172.21.15.17:0/2117598403 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184018bd0 con 0x7f71941021a0 2024-03-24T02:58:34.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.012+0000 7f719bd7e640 1 -- 172.21.15.17:0/2117598403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719419fe90 con 0x7f71941021a0 2024-03-24T02:58:34.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.013+0000 7f719bd7e640 1 -- 172.21.15.17:0/2117598403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71940fd640 con 0x7f71941021a0 2024-03-24T02:58:34.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.014+0000 7f7192ffd640 1 -- 172.21.15.17:0/2117598403 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f71840107d0 con 0x7f71941021a0 2024-03-24T02:58:34.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.015+0000 7f7192ffd640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f717c0a2880 0x7f717c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:34.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.015+0000 7f7199319640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f717c0a2880 0x7f717c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:34.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.015+0000 7f7192ffd640 1 -- 172.21.15.17:0/2117598403 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f71840cebd0 con 0x7f71941021a0 2024-03-24T02:58:34.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.015+0000 7f7199319640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f717c0a2880 0x7f717c0a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f718802c170 tx=0x7f718802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:34.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.019+0000 7f7192ffd640 1 -- 172.21.15.17:0/2117598403 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f718409c110 con 0x7f71941021a0 2024-03-24T02:58:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:34.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:58:34.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/468417140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:34.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:58:34.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1277252165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:34.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:33 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2979294787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:58:34.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.208+0000 7f719bd7e640 1 -- 172.21.15.17:0/2117598403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f719402d070 con 0x7f71941021a0 2024-03-24T02:58:34.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.208+0000 7f7192ffd640 1 -- 172.21.15.17:0/2117598403 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f718409ba10 con 0x7f71941021a0 2024-03-24T02:58:34.208 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:34.208 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-24T02:58:34.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T02:58:34.208 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-24T02:58:34.208 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-24T02:58:34.208 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T02:58:34.208 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-24T02:58:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.210+0000 7f719bd7e640 1 -- 172.21.15.17:0/2117598403 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f717c0a2880 msgr2=0x7f717c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.210+0000 7f719bd7e640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f717c0a2880 0x7f717c0a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f718802c170 tx=0x7f718802df90 comp rx=0 tx=0).stop 2024-03-24T02:58:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.210+0000 7f719bd7e640 1 -- 172.21.15.17:0/2117598403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71941021a0 msgr2=0x7f7194197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.210+0000 7f719bd7e640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71941021a0 0x7f7194197a10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f718400ebc0 tx=0x7f718400c6a0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.210+0000 7f719bd7e640 1 -- 172.21.15.17:0/2117598403 shutdown_connections 2024-03-24T02:58:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.210+0000 7f719bd7e640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f717c0a2880 0x7f717c0a4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.210+0000 7f719bd7e640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7194107400 0x7f719419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.210+0000 7f719bd7e640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7194104ad0 0x7f7194197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.210+0000 7f719bd7e640 1 --2- 172.21.15.17:0/2117598403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71941021a0 0x7f7194197a10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.210+0000 7f719bd7e640 1 -- 172.21.15.17:0/2117598403 >> 172.21.15.17:0/2117598403 conn(0x7f71940fbb50 msgr2=0x7f7194102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.211+0000 7f719bd7e640 1 -- 172.21.15.17:0/2117598403 shutdown_connections 2024-03-24T02:58:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.211+0000 7f719bd7e640 1 -- 172.21.15.17:0/2117598403 wait complete. 2024-03-24T02:58:34.221 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-24T02:58:34.221 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-24T02:58:34.249 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.249 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.249 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.249 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.250 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.250 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.250 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:34.250 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:58:34.250 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.250 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/468417140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:34.250 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:58:34.250 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1277252165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:34.250 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:33 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2979294787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:58:34.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:58:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/468417140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:58:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1277252165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:33 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2979294787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:58:34.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.331+0000 7f1139bf4640 1 -- 172.21.15.17:0/530351974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1134100f30 msgr2=0x7f11341013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:34.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.331+0000 7f1139bf4640 1 --2- 172.21.15.17:0/530351974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1134100f30 0x7f11341013b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f111c009900 tx=0x7f111c02bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:34.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.331+0000 7f1139bf4640 1 -- 172.21.15.17:0/530351974 shutdown_connections 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.331+0000 7f1139bf4640 1 --2- 172.21.15.17:0/530351974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11341018f0 0x7f1134108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.331+0000 7f1139bf4640 1 --2- 172.21.15.17:0/530351974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1134100f30 0x7f11341013b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.331+0000 7f1139bf4640 1 --2- 172.21.15.17:0/530351974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11340ffd40 0x7f1134100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.331+0000 7f1139bf4640 1 -- 172.21.15.17:0/530351974 >> 172.21.15.17:0/530351974 conn(0x7f11340fb2d0 msgr2=0x7f11340fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.331+0000 7f1139bf4640 1 -- 172.21.15.17:0/530351974 shutdown_connections 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.331+0000 7f1139bf4640 1 -- 172.21.15.17:0/530351974 wait complete. 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1139bf4640 1 Processor -- start 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1139bf4640 1 -- start start 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1139bf4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11340ffd40 0x7f11341096b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1139bf4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134100f30 0x7f113419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1139bf4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11341018f0 0x7f113419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1139bf4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113410c310 con 0x7f11341018f0 2024-03-24T02:58:34.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1139bf4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113410c190 con 0x7f1134100f30 2024-03-24T02:58:34.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1139bf4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113410c490 con 0x7f11340ffd40 2024-03-24T02:58:34.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1133fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11341018f0 0x7f113419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:34.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f11337fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11340ffd40 0x7f11341096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:34.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1133fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11341018f0 0x7f113419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58156/0 (socket says 172.21.15.17:58156) 2024-03-24T02:58:34.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1133fff640 1 -- 172.21.15.17:0/838794206 learned_addr learned my addr 172.21.15.17:0/838794206 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:34.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f112bfff640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134100f30 0x7f113419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:34.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1133fff640 1 -- 172.21.15.17:0/838794206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11340ffd40 msgr2=0x7f11341096b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:34.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1133fff640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11340ffd40 0x7f11341096b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1133fff640 1 -- 172.21.15.17:0/838794206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134100f30 msgr2=0x7f113419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:34.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1133fff640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134100f30 0x7f113419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:34.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1133fff640 1 -- 172.21.15.17:0/838794206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113419f6d0 con 0x7f11341018f0 2024-03-24T02:58:34.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f1133fff640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11341018f0 0x7f113419edc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f112c00b970 tx=0x7f112c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:34.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f11317fa640 1 -- 172.21.15.17:0/838794206 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c019070 con 0x7f11341018f0 2024-03-24T02:58:34.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.332+0000 7f11317fa640 1 -- 172.21.15.17:0/838794206 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f112c00cb40 con 0x7f11341018f0 2024-03-24T02:58:34.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.333+0000 7f1139bf4640 1 -- 172.21.15.17:0/838794206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113419f960 con 0x7f11341018f0 2024-03-24T02:58:34.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.333+0000 7f11317fa640 1 -- 172.21.15.17:0/838794206 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c007650 con 0x7f11341018f0 2024-03-24T02:58:34.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.333+0000 7f1139bf4640 1 -- 172.21.15.17:0/838794206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113419fe80 con 0x7f11341018f0 2024-03-24T02:58:34.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.334+0000 7f1139bf4640 1 -- 172.21.15.17:0/838794206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1134107060 con 0x7f11341018f0 2024-03-24T02:58:34.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.334+0000 7f11317fa640 1 -- 172.21.15.17:0/838794206 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f112c01c070 con 0x7f11341018f0 2024-03-24T02:58:34.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.335+0000 7f11317fa640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f11200a2880 0x7f11200a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:34.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.335+0000 7f11337fe640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f11200a2880 0x7f11200a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:34.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.335+0000 7f11317fa640 1 -- 172.21.15.17:0/838794206 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f112c0cdb10 con 0x7f11341018f0 2024-03-24T02:58:34.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.335+0000 7f11337fe640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f11200a2880 0x7f11200a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1134100d90 tx=0x7f1124005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:34.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.340+0000 7f11317fa640 1 -- 172.21.15.17:0/838794206 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f112c09aa00 con 0x7f11341018f0 2024-03-24T02:58:34.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:34.525+0000 7f1139bf4640 1 -- 172.21.15.17:0/838794206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f113402d190 con 0x7f11341018f0 2024-03-24T02:58:35.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.109+0000 7f11317fa640 1 -- 172.21.15.17:0/838794206 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f112c09eaa0 con 0x7f11341018f0 2024-03-24T02:58:35.109 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-24T02:58:35.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.112+0000 7f1139bf4640 1 -- 172.21.15.17:0/838794206 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f11200a2880 msgr2=0x7f11200a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:35.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.112+0000 7f1139bf4640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f11200a2880 0x7f11200a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1134100d90 tx=0x7f1124005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:35.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.112+0000 7f1139bf4640 1 -- 172.21.15.17:0/838794206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11341018f0 msgr2=0x7f113419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:35.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.112+0000 7f1139bf4640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11341018f0 0x7f113419edc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f112c00b970 tx=0x7f112c00be30 comp rx=0 tx=0).stop 2024-03-24T02:58:35.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.112+0000 7f1139bf4640 1 -- 172.21.15.17:0/838794206 shutdown_connections 2024-03-24T02:58:35.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.112+0000 7f1139bf4640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f11200a2880 0x7f11200a4d40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:35.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.112+0000 7f1139bf4640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11341018f0 0x7f113419edc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:35.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.112+0000 7f1139bf4640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134100f30 0x7f113419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:35.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.112+0000 7f1139bf4640 1 --2- 172.21.15.17:0/838794206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11340ffd40 0x7f11341096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:35.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.112+0000 7f1139bf4640 1 -- 172.21.15.17:0/838794206 >> 172.21.15.17:0/838794206 conn(0x7f11340fb2d0 msgr2=0x7f1134104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:35.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.112+0000 7f1139bf4640 1 -- 172.21.15.17:0/838794206 shutdown_connections 2024-03-24T02:58:35.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.112+0000 7f1139bf4640 1 -- 172.21.15.17:0/838794206 wait complete. 2024-03-24T02:58:35.124 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-24T02:58:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:34 smithi017 ceph-mon[31926]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:34 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2117598403' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-24T02:58:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:34 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/838794206' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-24T02:58:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:34 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:35.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:34 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:35.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.234+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/2559840879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e0102150 msgr2=0x7ff1e01089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:35.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.234+0000 7ff1e7fdf640 1 --2- 172.21.15.17:0/2559840879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e0102150 0x7ff1e01089e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009ca0 tx=0x7ff1d802c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:35.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.234+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/2559840879 shutdown_connections 2024-03-24T02:58:35.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.234+0000 7ff1e7fdf640 1 --2- 172.21.15.17:0/2559840879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e0102150 0x7ff1e01089e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:35.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.234+0000 7ff1e7fdf640 1 --2- 172.21.15.17:0/2559840879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1e0101790 0x7ff1e0101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:35.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.234+0000 7ff1e7fdf640 1 --2- 172.21.15.17:0/2559840879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e01005a0 0x7ff1e01009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:35.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.234+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/2559840879 >> 172.21.15.17:0/2559840879 conn(0x7ff1e00fbb30 msgr2=0x7ff1e00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:35.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.234+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/2559840879 shutdown_connections 2024-03-24T02:58:35.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.234+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/2559840879 wait complete. 2024-03-24T02:58:35.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e7fdf640 1 Processor -- start 2024-03-24T02:58:35.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e7fdf640 1 -- start start 2024-03-24T02:58:35.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e7fdf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e01005a0 0x7ff1e0069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:35.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e7fdf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e0101790 0x7ff1e019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:35.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e7fdf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1e0102150 0x7ff1e019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:35.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e7fdf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e010c230 con 0x7ff1e0101790 2024-03-24T02:58:35.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e7fdf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e010c0b0 con 0x7ff1e01005a0 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e7fdf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e010c3b0 con 0x7ff1e0102150 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e557a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e0101790 0x7ff1e019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e557a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e0101790 0x7ff1e019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58168/0 (socket says 172.21.15.17:58168) 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e557a640 1 -- 172.21.15.17:0/1789183565 learned_addr learned my addr 172.21.15.17:0/1789183565 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e5d7b640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e01005a0 0x7ff1e0069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e657c640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1e0102150 0x7ff1e019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e557a640 1 -- 172.21.15.17:0/1789183565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1e0102150 msgr2=0x7ff1e019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e557a640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1e0102150 0x7ff1e019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e557a640 1 -- 172.21.15.17:0/1789183565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e01005a0 msgr2=0x7ff1e0069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e557a640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e01005a0 0x7ff1e0069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.235+0000 7ff1e557a640 1 -- 172.21.15.17:0/1789183565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e019f5b0 con 0x7ff1e0101790 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.236+0000 7ff1e557a640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e0101790 0x7ff1e019c6b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e9f0 tx=0x7ff1dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.236+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1789183565 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00cd80 con 0x7ff1e0101790 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.236+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1789183565 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1dc005500 con 0x7ff1e0101790 2024-03-24T02:58:35.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.236+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/1789183565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e019f8a0 con 0x7ff1e0101790 2024-03-24T02:58:35.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.236+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1789183565 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc010690 con 0x7ff1e0101790 2024-03-24T02:58:35.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.236+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/1789183565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e019fdf0 con 0x7ff1e0101790 2024-03-24T02:58:35.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.237+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/1789183565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e01078c0 con 0x7ff1e0101790 2024-03-24T02:58:35.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.238+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1789183565 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff1dc0046c0 con 0x7ff1e0101790 2024-03-24T02:58:35.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.239+0000 7ff1d6ffd640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff1cc0a2880 0x7ff1cc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:35.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.239+0000 7ff1e5d7b640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff1cc0a2880 0x7ff1cc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:35.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.239+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1789183565 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7ff1dc0ceca0 con 0x7ff1e0101790 2024-03-24T02:58:35.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.239+0000 7ff1e5d7b640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff1cc0a2880 0x7ff1cc0a4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff1e01015f0 tx=0x7ff1d000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:35.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.242+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1789183565 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff1dc053740 con 0x7ff1e0101790 2024-03-24T02:58:35.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:34 smithi188 ceph-mon[36206]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:35.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:34 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2117598403' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-24T02:58:35.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:34 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/838794206' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-24T02:58:35.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:34 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:35.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:34 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:35.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:34 smithi019 ceph-mon[35944]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:35.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:34 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2117598403' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-24T02:58:35.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:34 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/838794206' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-24T02:58:35.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:34 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:35.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:34 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:35.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:35.431+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/1789183565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7ff1e002d070 con 0x7ff1e0101790 2024-03-24T02:58:36.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.111+0000 7ff1d6ffd640 1 -- 172.21.15.17:0/1789183565 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7ff1dc09c070 con 0x7ff1e0101790 2024-03-24T02:58:36.111 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-24T02:58:36.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.114+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/1789183565 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff1cc0a2880 msgr2=0x7ff1cc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:36.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.114+0000 7ff1e7fdf640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff1cc0a2880 0x7ff1cc0a4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff1e01015f0 tx=0x7ff1d000b390 comp rx=0 tx=0).stop 2024-03-24T02:58:36.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.114+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/1789183565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e0101790 msgr2=0x7ff1e019c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:36.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.114+0000 7ff1e7fdf640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e0101790 0x7ff1e019c6b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e9f0 tx=0x7ff1dc00edf0 comp rx=0 tx=0).stop 2024-03-24T02:58:36.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.114+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/1789183565 shutdown_connections 2024-03-24T02:58:36.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.114+0000 7ff1e7fdf640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff1cc0a2880 0x7ff1cc0a4d40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:36.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.114+0000 7ff1e7fdf640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1e0102150 0x7ff1e019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:36.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.114+0000 7ff1e7fdf640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1e0101790 0x7ff1e019c6b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:36.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.114+0000 7ff1e7fdf640 1 --2- 172.21.15.17:0/1789183565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e01005a0 0x7ff1e0069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:36.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.114+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/1789183565 >> 172.21.15.17:0/1789183565 conn(0x7ff1e00fbb30 msgr2=0x7ff1e0104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:36.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.114+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/1789183565 shutdown_connections 2024-03-24T02:58:36.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.114+0000 7ff1e7fdf640 1 -- 172.21.15.17:0/1789183565 wait complete. 2024-03-24T02:58:36.134 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-24T02:58:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:58:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:35 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/838794206' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-24T02:58:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:35 smithi017 ceph-mon[31926]: osdmap e82: 12 total, 12 up, 12 in 2024-03-24T02:58:36.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:35 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1789183565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.242+0000 7fa1cf844640 1 -- 172.21.15.17:0/3299068004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1c8102160 msgr2=0x7fa1c81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.242+0000 7fa1cf844640 1 --2- 172.21.15.17:0/3299068004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1c8102160 0x7fa1c81089f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009ca0 tx=0x7fa1c002c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.242+0000 7fa1cf844640 1 -- 172.21.15.17:0/3299068004 shutdown_connections 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.242+0000 7fa1cf844640 1 --2- 172.21.15.17:0/3299068004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1c8102160 0x7fa1c81089f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.242+0000 7fa1cf844640 1 --2- 172.21.15.17:0/3299068004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1c81017a0 0x7fa1c8101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.242+0000 7fa1cf844640 1 --2- 172.21.15.17:0/3299068004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1c81005b0 0x7fa1c81009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.242+0000 7fa1cf844640 1 -- 172.21.15.17:0/3299068004 >> 172.21.15.17:0/3299068004 conn(0x7fa1c80fbb40 msgr2=0x7fa1c80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.242+0000 7fa1cf844640 1 -- 172.21.15.17:0/3299068004 shutdown_connections 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.242+0000 7fa1cf844640 1 -- 172.21.15.17:0/3299068004 wait complete. 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1cf844640 1 Processor -- start 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1cf844640 1 -- start start 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1cf844640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1c81005b0 0x7fa1c810c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1cf844640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1c81017a0 0x7fa1c810a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1cf844640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1c8102160 0x7fa1c810ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1cf844640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c810d320 con 0x7fa1c81017a0 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1cf844640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c810d1a0 con 0x7fa1c81005b0 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1cf844640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c810d4a0 con 0x7fa1c8102160 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1ccddf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1c81017a0 0x7fa1c810a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1ccddf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1c81017a0 0x7fa1c810a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58178/0 (socket says 172.21.15.17:58178) 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1ccddf640 1 -- 172.21.15.17:0/1657138701 learned_addr learned my addr 172.21.15.17:0/1657138701 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1cd5e0640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1c81005b0 0x7fa1c810c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.243+0000 7fa1cdde1640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1c8102160 0x7fa1c810ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.244+0000 7fa1ccddf640 1 -- 172.21.15.17:0/1657138701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1c8102160 msgr2=0x7fa1c810ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.244+0000 7fa1ccddf640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1c8102160 0x7fa1c810ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.244+0000 7fa1ccddf640 1 -- 172.21.15.17:0/1657138701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1c81005b0 msgr2=0x7fa1c810c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.244+0000 7fa1ccddf640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1c81005b0 0x7fa1c810c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.244+0000 7fa1ccddf640 1 -- 172.21.15.17:0/1657138701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c810b6c0 con 0x7fa1c81017a0 2024-03-24T02:58:36.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.244+0000 7fa1cd5e0640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1c81005b0 0x7fa1c810c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:36.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.244+0000 7fa1ccddf640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1c81017a0 0x7fa1c810a8f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800e9f0 tx=0x7fa1b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:36.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.244+0000 7fa1be7fc640 1 -- 172.21.15.17:0/1657138701 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b800cd80 con 0x7fa1c81017a0 2024-03-24T02:58:36.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.244+0000 7fa1cf844640 1 -- 172.21.15.17:0/1657138701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c81a19f0 con 0x7fa1c81017a0 2024-03-24T02:58:36.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.244+0000 7fa1cf844640 1 -- 172.21.15.17:0/1657138701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c81a1f90 con 0x7fa1c81017a0 2024-03-24T02:58:36.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.244+0000 7fa1be7fc640 1 -- 172.21.15.17:0/1657138701 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1b8010910 con 0x7fa1c81017a0 2024-03-24T02:58:36.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.244+0000 7fa1be7fc640 1 -- 172.21.15.17:0/1657138701 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8018950 con 0x7fa1c81017a0 2024-03-24T02:58:36.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.245+0000 7fa1cf844640 1 -- 172.21.15.17:0/1657138701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c8107870 con 0x7fa1c81017a0 2024-03-24T02:58:36.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.246+0000 7fa1be7fc640 1 -- 172.21.15.17:0/1657138701 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa1b8010ac0 con 0x7fa1c81017a0 2024-03-24T02:58:36.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.247+0000 7fa1be7fc640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa1b40a2800 0x7fa1b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:36.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.247+0000 7fa1cd5e0640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa1b40a2800 0x7fa1b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:36.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.247+0000 7fa1be7fc640 1 -- 172.21.15.17:0/1657138701 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fa1b80cf1f0 con 0x7fa1c81017a0 2024-03-24T02:58:36.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.247+0000 7fa1cd5e0640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa1b40a2800 0x7fa1b40a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8101600 tx=0x7fa1c400a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:36.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.251+0000 7fa1be7fc640 1 -- 172.21.15.17:0/1657138701 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa1b809c420 con 0x7fa1c81017a0 2024-03-24T02:58:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:58:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:35 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/838794206' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-24T02:58:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:35 smithi188 ceph-mon[36206]: osdmap e82: 12 total, 12 up, 12 in 2024-03-24T02:58:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:35 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1789183565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-24T02:58:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T02:58:36.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T02:58:36.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:58:36.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:35 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/838794206' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-24T02:58:36.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:35 smithi019 ceph-mon[35944]: osdmap e82: 12 total, 12 up, 12 in 2024-03-24T02:58:36.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:35 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1789183565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-24T02:58:36.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:36.474+0000 7fa1cf844640 1 -- 172.21.15.17:0/1657138701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa1c802d070 con 0x7fa1c81017a0 2024-03-24T02:58:37.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:36 smithi017 ceph-mon[31926]: pgmap v285: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:37.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:36 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1789183565' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-24T02:58:37.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:36 smithi017 ceph-mon[31926]: osdmap e83: 12 total, 12 up, 12 in 2024-03-24T02:58:37.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:36 smithi017 ceph-mon[31926]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T02:58:37.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:36 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1657138701' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-24T02:58:37.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:36 smithi017 ceph-mon[31926]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-24T02:58:37.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:36 smithi188 ceph-mon[36206]: pgmap v285: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:37.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:36 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1789183565' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-24T02:58:37.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:36 smithi188 ceph-mon[36206]: osdmap e83: 12 total, 12 up, 12 in 2024-03-24T02:58:37.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:36 smithi188 ceph-mon[36206]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T02:58:37.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:36 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1657138701' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-24T02:58:37.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:36 smithi188 ceph-mon[36206]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-24T02:58:37.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:36 smithi019 ceph-mon[35944]: pgmap v285: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:37.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:36 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1789183565' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-24T02:58:37.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:36 smithi019 ceph-mon[35944]: osdmap e83: 12 total, 12 up, 12 in 2024-03-24T02:58:37.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:36 smithi019 ceph-mon[35944]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T02:58:37.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:36 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1657138701' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-24T02:58:37.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:36 smithi019 ceph-mon[35944]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-24T02:58:37.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.370+0000 7fa1be7fc640 1 -- 172.21.15.17:0/1657138701 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa1b809bd20 con 0x7fa1c81017a0 2024-03-24T02:58:37.369 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 2 and data pool 3 2024-03-24T02:58:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.373+0000 7fa1cf844640 1 -- 172.21.15.17:0/1657138701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa1b40a2800 msgr2=0x7fa1b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.373+0000 7fa1cf844640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa1b40a2800 0x7fa1b40a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8101600 tx=0x7fa1c400a5e0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.373+0000 7fa1cf844640 1 -- 172.21.15.17:0/1657138701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1c81017a0 msgr2=0x7fa1c810a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.373+0000 7fa1cf844640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1c81017a0 0x7fa1c810a8f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800e9f0 tx=0x7fa1b800eeb0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.373+0000 7fa1cf844640 1 -- 172.21.15.17:0/1657138701 shutdown_connections 2024-03-24T02:58:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.373+0000 7fa1cf844640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa1b40a2800 0x7fa1b40a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.373+0000 7fa1cf844640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1c8102160 0x7fa1c810ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.373+0000 7fa1cf844640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1c81017a0 0x7fa1c810a8f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.373+0000 7fa1cf844640 1 --2- 172.21.15.17:0/1657138701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1c81005b0 0x7fa1c810c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.373+0000 7fa1cf844640 1 -- 172.21.15.17:0/1657138701 >> 172.21.15.17:0/1657138701 conn(0x7fa1c80fbb40 msgr2=0x7fa1c8104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:37.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.373+0000 7fa1cf844640 1 -- 172.21.15.17:0/1657138701 shutdown_connections 2024-03-24T02:58:37.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.373+0000 7fa1cf844640 1 -- 172.21.15.17:0/1657138701 wait complete. 2024-03-24T02:58:37.388 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-24T02:58:37.388 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-24T02:58:37.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.503+0000 7fb1655f3640 1 -- 172.21.15.17:0/1197355684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb160102160 msgr2=0x7fb1601089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:37.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.503+0000 7fb1655f3640 1 --2- 172.21.15.17:0/1197355684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb160102160 0x7fb1601089f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb158009ca0 tx=0x7fb15802c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.503+0000 7fb1655f3640 1 -- 172.21.15.17:0/1197355684 shutdown_connections 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.503+0000 7fb1655f3640 1 --2- 172.21.15.17:0/1197355684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb160102160 0x7fb1601089f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.503+0000 7fb1655f3640 1 --2- 172.21.15.17:0/1197355684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1601017a0 0x7fb160101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.503+0000 7fb1655f3640 1 --2- 172.21.15.17:0/1197355684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1601005b0 0x7fb1601009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.503+0000 7fb1655f3640 1 -- 172.21.15.17:0/1197355684 >> 172.21.15.17:0/1197355684 conn(0x7fb1600fbb40 msgr2=0x7fb1600fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.503+0000 7fb1655f3640 1 -- 172.21.15.17:0/1197355684 shutdown_connections 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.503+0000 7fb1655f3640 1 -- 172.21.15.17:0/1197355684 wait complete. 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb1655f3640 1 Processor -- start 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb1655f3640 1 -- start start 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb1655f3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1601005b0 0x7fb1601978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb1655f3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1601017a0 0x7fb160197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb1655f3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb160102160 0x7fb16019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb1655f3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16010d350 con 0x7fb160102160 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb1655f3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16010d1d0 con 0x7fb1601005b0 2024-03-24T02:58:37.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb1655f3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16010d4d0 con 0x7fb1601017a0 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb15f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb160102160 0x7fb16019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb15f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb160102160 0x7fb16019ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58198/0 (socket says 172.21.15.17:58198) 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb15f7fe640 1 -- 172.21.15.17:0/82601205 learned_addr learned my addr 172.21.15.17:0/82601205 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb157fff640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1601017a0 0x7fb160197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb15effd640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1601005b0 0x7fb1601978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb15f7fe640 1 -- 172.21.15.17:0/82601205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1601017a0 msgr2=0x7fb160197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb15f7fe640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1601017a0 0x7fb160197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb15f7fe640 1 -- 172.21.15.17:0/82601205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1601005b0 msgr2=0x7fb1601978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb15f7fe640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1601005b0 0x7fb1601978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb15f7fe640 1 -- 172.21.15.17:0/82601205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16019f4f0 con 0x7fb160102160 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb157fff640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1601017a0 0x7fb160197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb15effd640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1601005b0 0x7fb1601978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.504+0000 7fb15f7fe640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb160102160 0x7fb16019ee80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb15802c700 tx=0x7fb158032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:37.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.505+0000 7fb15cff9640 1 -- 172.21.15.17:0/82601205 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158039040 con 0x7fb160102160 2024-03-24T02:58:37.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.505+0000 7fb1655f3640 1 -- 172.21.15.17:0/82601205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16019f780 con 0x7fb160102160 2024-03-24T02:58:37.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.505+0000 7fb15cff9640 1 -- 172.21.15.17:0/82601205 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb158044070 con 0x7fb160102160 2024-03-24T02:58:37.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.505+0000 7fb15cff9640 1 -- 172.21.15.17:0/82601205 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1580042a0 con 0x7fb160102160 2024-03-24T02:58:37.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.505+0000 7fb1655f3640 1 -- 172.21.15.17:0/82601205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16019fcc0 con 0x7fb160102160 2024-03-24T02:58:37.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.506+0000 7fb1655f3640 1 -- 172.21.15.17:0/82601205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1601078d0 con 0x7fb160102160 2024-03-24T02:58:37.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.507+0000 7fb15cff9640 1 -- 172.21.15.17:0/82601205 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb158003970 con 0x7fb160102160 2024-03-24T02:58:37.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.507+0000 7fb15cff9640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb14c0a2800 0x7fb14c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:37.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.507+0000 7fb15effd640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb14c0a2800 0x7fb14c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:37.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.507+0000 7fb15cff9640 1 -- 172.21.15.17:0/82601205 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb1580f0f20 con 0x7fb160102160 2024-03-24T02:58:37.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.508+0000 7fb15effd640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb14c0a2800 0x7fb14c0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb150005d00 tx=0x7fb150005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:37.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.512+0000 7fb15cff9640 1 -- 172.21.15.17:0/82601205 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb1580be1a0 con 0x7fb160102160 2024-03-24T02:58:37.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.703+0000 7fb1655f3640 1 -- 172.21.15.17:0/82601205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fb1600622a0 con 0x7fb160102160 2024-03-24T02:58:37.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.957+0000 7fb15cff9640 1 -- 172.21.15.17:0/82601205 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb1580bdaa0 con 0x7fb160102160 2024-03-24T02:58:37.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.960+0000 7fb1655f3640 1 -- 172.21.15.17:0/82601205 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb14c0a2800 msgr2=0x7fb14c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:37.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.960+0000 7fb1655f3640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb14c0a2800 0x7fb14c0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb150005d00 tx=0x7fb150005ef0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.960+0000 7fb1655f3640 1 -- 172.21.15.17:0/82601205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb160102160 msgr2=0x7fb16019ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:37.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.960+0000 7fb1655f3640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb160102160 0x7fb16019ee80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb15802c700 tx=0x7fb158032f20 comp rx=0 tx=0).stop 2024-03-24T02:58:37.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.960+0000 7fb1655f3640 1 -- 172.21.15.17:0/82601205 shutdown_connections 2024-03-24T02:58:37.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.960+0000 7fb1655f3640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb14c0a2800 0x7fb14c0a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.960+0000 7fb1655f3640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb160102160 0x7fb16019ee80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.960+0000 7fb1655f3640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1601017a0 0x7fb160197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.960+0000 7fb1655f3640 1 --2- 172.21.15.17:0/82601205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1601005b0 0x7fb1601978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:37.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.960+0000 7fb1655f3640 1 -- 172.21.15.17:0/82601205 >> 172.21.15.17:0/82601205 conn(0x7fb1600fbb40 msgr2=0x7fb160104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:37.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.960+0000 7fb1655f3640 1 -- 172.21.15.17:0/82601205 shutdown_connections 2024-03-24T02:58:37.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:37.960+0000 7fb1655f3640 1 -- 172.21.15.17:0/82601205 wait complete. 2024-03-24T02:58:37.972 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-24T02:58:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.085+0000 7f523c061640 1 -- 172.21.15.17:0/1849245923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52341017d0 msgr2=0x7f5234101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.085+0000 7f523c061640 1 --2- 172.21.15.17:0/1849245923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52341017d0 0x7f5234101c50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5224009900 tx=0x7f522402bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.085+0000 7f523c061640 1 -- 172.21.15.17:0/1849245923 shutdown_connections 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.085+0000 7f523c061640 1 --2- 172.21.15.17:0/1849245923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5234102190 0x7f5234108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.085+0000 7f523c061640 1 --2- 172.21.15.17:0/1849245923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52341017d0 0x7f5234101c50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.085+0000 7f523c061640 1 --2- 172.21.15.17:0/1849245923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52341005e0 0x7f5234100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.085+0000 7f523c061640 1 -- 172.21.15.17:0/1849245923 >> 172.21.15.17:0/1849245923 conn(0x7f52340fbb50 msgr2=0x7f52340fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.085+0000 7f523c061640 1 -- 172.21.15.17:0/1849245923 shutdown_connections 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.085+0000 7f523c061640 1 -- 172.21.15.17:0/1849245923 wait complete. 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f523c061640 1 Processor -- start 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f523c061640 1 -- start start 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f523c061640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52341005e0 0x7f5234199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f523c061640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52341017d0 0x7f523419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f523c061640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5234102190 0x7f52341a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f523c061640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523410c260 con 0x7f52341017d0 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f523c061640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523410c0e0 con 0x7f5234102190 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f523c061640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523410c3e0 con 0x7f52341005e0 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f52395fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52341017d0 0x7f523419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f52395fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52341017d0 0x7f523419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58210/0 (socket says 172.21.15.17:58210) 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f52395fc640 1 -- 172.21.15.17:0/464387722 learned_addr learned my addr 172.21.15.17:0/464387722 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f5239dfd640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52341005e0 0x7f5234199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f523a5fe640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5234102190 0x7f52341a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f52395fc640 1 -- 172.21.15.17:0/464387722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52341005e0 msgr2=0x7f5234199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f52395fc640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52341005e0 0x7f5234199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f52395fc640 1 -- 172.21.15.17:0/464387722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5234102190 msgr2=0x7f52341a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f52395fc640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5234102190 0x7f52341a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.086+0000 7f52395fc640 1 -- 172.21.15.17:0/464387722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52341a1830 con 0x7f52341017d0 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.087+0000 7f52395fc640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52341017d0 0x7f523419a160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5224009230 tx=0x7f5224005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.087+0000 7f522affd640 1 -- 172.21.15.17:0/464387722 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224041070 con 0x7f52341017d0 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.087+0000 7f522affd640 1 -- 172.21.15.17:0/464387722 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52240054e0 con 0x7f52341017d0 2024-03-24T02:58:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.087+0000 7f523c061640 1 -- 172.21.15.17:0/464387722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52341a1ac0 con 0x7f52341017d0 2024-03-24T02:58:38.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.087+0000 7f522affd640 1 -- 172.21.15.17:0/464387722 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224034440 con 0x7f52341017d0 2024-03-24T02:58:38.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.087+0000 7f523c061640 1 -- 172.21.15.17:0/464387722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52341a2000 con 0x7f52341017d0 2024-03-24T02:58:38.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.088+0000 7f523c061640 1 -- 172.21.15.17:0/464387722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52341a21d0 con 0x7f52341017d0 2024-03-24T02:58:38.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.089+0000 7f522affd640 1 -- 172.21.15.17:0/464387722 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5224044070 con 0x7f52341017d0 2024-03-24T02:58:38.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.089+0000 7f522affd640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f52200a2880 0x7f52200a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:38.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.089+0000 7f522affd640 1 -- 172.21.15.17:0/464387722 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f52240f07a0 con 0x7f52341017d0 2024-03-24T02:58:38.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.089+0000 7f5239dfd640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f52200a2880 0x7f52200a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:38.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.090+0000 7f5239dfd640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f52200a2880 0x7f52200a4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f523419b0f0 tx=0x7f523000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:38.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.094+0000 7f522affd640 1 -- 172.21.15.17:0/464387722 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f52240bd810 con 0x7f52341017d0 2024-03-24T02:58:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:38.278+0000 7f523c061640 1 -- 172.21.15.17:0/464387722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f52341a26c0 con 0x7f52341017d0 2024-03-24T02:58:38.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:37 smithi188 ceph-mon[36206]: osdmap e84: 12 total, 12 up, 12 in 2024-03-24T02:58:38.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:37 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1657138701' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-24T02:58:38.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:37 smithi188 ceph-mon[36206]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-24T02:58:38.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:37 smithi188 ceph-mon[36206]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-24T02:58:38.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:37 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/82601205' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T02:58:38.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:37 smithi188 ceph-mon[36206]: pgmap v288: 97 pgs: 17 creating+peering, 21 active+clean, 59 unknown; 578 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-03-24T02:58:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:37 smithi019 ceph-mon[35944]: osdmap e84: 12 total, 12 up, 12 in 2024-03-24T02:58:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:37 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1657138701' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-24T02:58:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:37 smithi019 ceph-mon[35944]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-24T02:58:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:37 smithi019 ceph-mon[35944]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-24T02:58:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:37 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/82601205' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T02:58:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:37 smithi019 ceph-mon[35944]: pgmap v288: 97 pgs: 17 creating+peering, 21 active+clean, 59 unknown; 578 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-03-24T02:58:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:37 smithi017 ceph-mon[31926]: osdmap e84: 12 total, 12 up, 12 in 2024-03-24T02:58:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:37 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1657138701' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-24T02:58:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:37 smithi017 ceph-mon[31926]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-24T02:58:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:37 smithi017 ceph-mon[31926]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-24T02:58:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:37 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/82601205' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T02:58:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:37 smithi017 ceph-mon[31926]: pgmap v288: 97 pgs: 17 creating+peering, 21 active+clean, 59 unknown; 578 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-03-24T02:58:39.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:38 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/82601205' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-24T02:58:39.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:38 smithi188 ceph-mon[36206]: osdmap e85: 12 total, 12 up, 12 in 2024-03-24T02:58:39.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:38 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/464387722' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-24T02:58:39.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:38 smithi188 ceph-mon[36206]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T02:58:39.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:38 smithi188 ceph-mon[36206]: Cluster is now healthy 2024-03-24T02:58:39.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:38 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] up:active 2024-03-24T02:58:39.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:38 smithi188 ceph-mon[36206]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:39.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:38 smithi188 ceph-mon[36206]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:39.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:38 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/82601205' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-24T02:58:39.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:38 smithi019 ceph-mon[35944]: osdmap e85: 12 total, 12 up, 12 in 2024-03-24T02:58:39.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:38 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/464387722' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-24T02:58:39.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:38 smithi019 ceph-mon[35944]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T02:58:39.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:38 smithi019 ceph-mon[35944]: Cluster is now healthy 2024-03-24T02:58:39.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:38 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] up:active 2024-03-24T02:58:39.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:38 smithi019 ceph-mon[35944]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:39.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:38 smithi019 ceph-mon[35944]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:39.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:38 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/82601205' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-24T02:58:39.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:38 smithi017 ceph-mon[31926]: osdmap e85: 12 total, 12 up, 12 in 2024-03-24T02:58:39.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:38 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/464387722' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-24T02:58:39.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:38 smithi017 ceph-mon[31926]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T02:58:39.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:38 smithi017 ceph-mon[31926]: Cluster is now healthy 2024-03-24T02:58:39.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:38 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] up:active 2024-03-24T02:58:39.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:38 smithi017 ceph-mon[31926]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:39.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:38 smithi017 ceph-mon[31926]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:39.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.972+0000 7f522affd640 1 -- 172.21.15.17:0/464387722 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7f52240bd110 con 0x7f52341017d0 2024-03-24T02:58:39.972 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data_ec' created 2024-03-24T02:58:39.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.976+0000 7f523c061640 1 -- 172.21.15.17:0/464387722 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f52200a2880 msgr2=0x7f52200a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:39.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.976+0000 7f523c061640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f52200a2880 0x7f52200a4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f523419b0f0 tx=0x7f523000b390 comp rx=0 tx=0).stop 2024-03-24T02:58:39.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.976+0000 7f523c061640 1 -- 172.21.15.17:0/464387722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52341017d0 msgr2=0x7f523419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:39.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.976+0000 7f523c061640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52341017d0 0x7f523419a160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5224009230 tx=0x7f5224005dd0 comp rx=0 tx=0).stop 2024-03-24T02:58:39.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.976+0000 7f523c061640 1 -- 172.21.15.17:0/464387722 shutdown_connections 2024-03-24T02:58:39.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.976+0000 7f523c061640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f52200a2880 0x7f52200a4d40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:39.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.976+0000 7f523c061640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5234102190 0x7f52341a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:39.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.976+0000 7f523c061640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52341017d0 0x7f523419a160 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:39.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.976+0000 7f523c061640 1 --2- 172.21.15.17:0/464387722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52341005e0 0x7f5234199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:39.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.976+0000 7f523c061640 1 -- 172.21.15.17:0/464387722 >> 172.21.15.17:0/464387722 conn(0x7f52340fbb50 msgr2=0x7f52340fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:39.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.976+0000 7f523c061640 1 -- 172.21.15.17:0/464387722 shutdown_connections 2024-03-24T02:58:39.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:39.976+0000 7f523c061640 1 -- 172.21.15.17:0/464387722 wait complete. 2024-03-24T02:58:39.995 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-24T02:58:40.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.107+0000 7f395bcb2640 1 -- 172.21.15.17:0/1841951123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3954101210 msgr2=0x7f3954101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:40.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.107+0000 7f395bcb2640 1 --2- 172.21.15.17:0/1841951123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3954101210 0x7f3954101690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f394c009ca0 tx=0x7f394c02c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:40.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.107+0000 7f395bcb2640 1 -- 172.21.15.17:0/1841951123 shutdown_connections 2024-03-24T02:58:40.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.107+0000 7f395bcb2640 1 --2- 172.21.15.17:0/1841951123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3954101210 0x7f3954101690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:40.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.107+0000 7f395bcb2640 1 --2- 172.21.15.17:0/1841951123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3954100870 0x7f3954100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:40.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.107+0000 7f395bcb2640 1 --2- 172.21.15.17:0/1841951123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39540fff10 0x7f3954100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:40.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.107+0000 7f395bcb2640 1 -- 172.21.15.17:0/1841951123 >> 172.21.15.17:0/1841951123 conn(0x7f39540fbb30 msgr2=0x7f39540fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:40.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395bcb2640 1 -- 172.21.15.17:0/1841951123 shutdown_connections 2024-03-24T02:58:40.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395bcb2640 1 -- 172.21.15.17:0/1841951123 wait complete. 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395bcb2640 1 Processor -- start 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395bcb2640 1 -- start start 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395bcb2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39540fff10 0x7f3954199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395bcb2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3954100870 0x7f395419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395bcb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3954101210 0x7f39541a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395bcb2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395410f4f0 con 0x7f3954100870 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395bcb2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395410f370 con 0x7f3954101210 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395bcb2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395410f670 con 0x7f39540fff10 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395924d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3954100870 0x7f395419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395924d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3954100870 0x7f395419a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58236/0 (socket says 172.21.15.17:58236) 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395924d640 1 -- 172.21.15.17:0/538390550 learned_addr learned my addr 172.21.15.17:0/538390550 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f3959a4e640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39540fff10 0x7f3954199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.108+0000 7f395a24f640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3954101210 0x7f39541a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:40.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.109+0000 7f395924d640 1 -- 172.21.15.17:0/538390550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39540fff10 msgr2=0x7f3954199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:40.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.109+0000 7f395924d640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39540fff10 0x7f3954199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:40.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.109+0000 7f395924d640 1 -- 172.21.15.17:0/538390550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3954101210 msgr2=0x7f39541a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:40.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.109+0000 7f395924d640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3954101210 0x7f39541a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:40.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.109+0000 7f395924d640 1 -- 172.21.15.17:0/538390550 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39541a1790 con 0x7f3954100870 2024-03-24T02:58:40.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.109+0000 7f395924d640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3954100870 0x7f395419a0b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f394400ea60 tx=0x7f394400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:40.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.109+0000 7f394affd640 1 -- 172.21.15.17:0/538390550 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394400cde0 con 0x7f3954100870 2024-03-24T02:58:40.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.109+0000 7f394affd640 1 -- 172.21.15.17:0/538390550 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39440035f0 con 0x7f3954100870 2024-03-24T02:58:40.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.109+0000 7f395bcb2640 1 -- 172.21.15.17:0/538390550 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39541a1a80 con 0x7f3954100870 2024-03-24T02:58:40.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.109+0000 7f394affd640 1 -- 172.21.15.17:0/538390550 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944010670 con 0x7f3954100870 2024-03-24T02:58:40.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.109+0000 7f395bcb2640 1 -- 172.21.15.17:0/538390550 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39541a2020 con 0x7f3954100870 2024-03-24T02:58:40.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.110+0000 7f395bcb2640 1 -- 172.21.15.17:0/538390550 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395410ae10 con 0x7f3954100870 2024-03-24T02:58:40.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.111+0000 7f394affd640 1 -- 172.21.15.17:0/538390550 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f39440037a0 con 0x7f3954100870 2024-03-24T02:58:40.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.112+0000 7f394affd640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f39400a2880 0x7f39400a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:40.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.112+0000 7f394affd640 1 -- 172.21.15.17:0/538390550 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f39440cf040 con 0x7f3954100870 2024-03-24T02:58:40.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.112+0000 7f3959a4e640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f39400a2880 0x7f39400a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:40.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.112+0000 7f3959a4e640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f39400a2880 0x7f39400a4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3950005d00 tx=0x7f395000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:40.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.116+0000 7f394affd640 1 -- 172.21.15.17:0/538390550 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f394409bef0 con 0x7f3954100870 2024-03-24T02:58:40.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:39 smithi188 ceph-mon[36206]: osdmap e86: 12 total, 12 up, 12 in 2024-03-24T02:58:40.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:39 smithi188 ceph-mon[36206]: pgmap v291: 97 pgs: 17 creating+peering, 70 active+clean, 10 unknown; 578 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-03-24T02:58:40.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.300+0000 7f395bcb2640 1 -- 172.21.15.17:0/538390550 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f395402d070 con 0x7f3954100870 2024-03-24T02:58:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:39 smithi019 ceph-mon[35944]: osdmap e86: 12 total, 12 up, 12 in 2024-03-24T02:58:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:39 smithi019 ceph-mon[35944]: pgmap v291: 97 pgs: 17 creating+peering, 70 active+clean, 10 unknown; 578 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-03-24T02:58:40.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:39 smithi017 ceph-mon[31926]: osdmap e86: 12 total, 12 up, 12 in 2024-03-24T02:58:40.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:39 smithi017 ceph-mon[31926]: pgmap v291: 97 pgs: 17 creating+peering, 70 active+clean, 10 unknown; 578 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-03-24T02:58:40.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.974+0000 7f394affd640 1 -- 172.21.15.17:0/538390550 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7f394409b7f0 con 0x7f3954100870 2024-03-24T02:58:40.974 INFO:teuthology.orchestra.run.smithi017.stderr:set pool 4 allow_ec_overwrites to true 2024-03-24T02:58:40.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.978+0000 7f395bcb2640 1 -- 172.21.15.17:0/538390550 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f39400a2880 msgr2=0x7f39400a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:40.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.978+0000 7f395bcb2640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f39400a2880 0x7f39400a4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3950005d00 tx=0x7f395000b410 comp rx=0 tx=0).stop 2024-03-24T02:58:40.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.978+0000 7f395bcb2640 1 -- 172.21.15.17:0/538390550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3954100870 msgr2=0x7f395419a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:40.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.978+0000 7f395bcb2640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3954100870 0x7f395419a0b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f394400ea60 tx=0x7f394400ef20 comp rx=0 tx=0).stop 2024-03-24T02:58:40.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.978+0000 7f395bcb2640 1 -- 172.21.15.17:0/538390550 shutdown_connections 2024-03-24T02:58:40.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.978+0000 7f395bcb2640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f39400a2880 0x7f39400a4d40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:40.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.978+0000 7f395bcb2640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3954101210 0x7f39541a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:40.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.978+0000 7f395bcb2640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3954100870 0x7f395419a0b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:40.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.978+0000 7f395bcb2640 1 --2- 172.21.15.17:0/538390550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39540fff10 0x7f3954199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:40.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.978+0000 7f395bcb2640 1 -- 172.21.15.17:0/538390550 >> 172.21.15.17:0/538390550 conn(0x7f39540fbb30 msgr2=0x7f39540fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:40.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.978+0000 7f395bcb2640 1 -- 172.21.15.17:0/538390550 shutdown_connections 2024-03-24T02:58:40.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:40.978+0000 7f395bcb2640 1 -- 172.21.15.17:0/538390550 wait complete. 2024-03-24T02:58:41.002 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-24T02:58:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.113+0000 7ff94cb77640 1 -- 172.21.15.17:0/969063696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff948102150 msgr2=0x7ff9481089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.113+0000 7ff94cb77640 1 --2- 172.21.15.17:0/969063696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff948102150 0x7ff9481089e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff938009900 tx=0x7ff93802bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.113+0000 7ff94cb77640 1 -- 172.21.15.17:0/969063696 shutdown_connections 2024-03-24T02:58:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.113+0000 7ff94cb77640 1 --2- 172.21.15.17:0/969063696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff948102150 0x7ff9481089e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.113+0000 7ff94cb77640 1 --2- 172.21.15.17:0/969063696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff948101790 0x7ff948101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.113+0000 7ff94cb77640 1 --2- 172.21.15.17:0/969063696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9481005a0 0x7ff9481009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.113+0000 7ff94cb77640 1 -- 172.21.15.17:0/969063696 >> 172.21.15.17:0/969063696 conn(0x7ff9480fbb30 msgr2=0x7ff9480fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.113+0000 7ff94cb77640 1 -- 172.21.15.17:0/969063696 shutdown_connections 2024-03-24T02:58:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.114+0000 7ff94cb77640 1 -- 172.21.15.17:0/969063696 wait complete. 2024-03-24T02:58:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.114+0000 7ff94cb77640 1 Processor -- start 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.114+0000 7ff94cb77640 1 -- start start 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.114+0000 7ff94cb77640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9481005a0 0x7ff948197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.114+0000 7ff94cb77640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff948101790 0x7ff948197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.114+0000 7ff94cb77640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff948102150 0x7ff94819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.114+0000 7ff94cb77640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94810c230 con 0x7ff9481005a0 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.114+0000 7ff94cb77640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94810c0b0 con 0x7ff948102150 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.114+0000 7ff94cb77640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94810c3b0 con 0x7ff948101790 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.114+0000 7ff94659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9481005a0 0x7ff948197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.114+0000 7ff94659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9481005a0 0x7ff948197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38834/0 (socket says 172.21.15.17:38834) 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.114+0000 7ff94659c640 1 -- 172.21.15.17:0/1139261541 learned_addr learned my addr 172.21.15.17:0/1139261541 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff945d9b640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff948101790 0x7ff948197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff94659c640 1 -- 172.21.15.17:0/1139261541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff948101790 msgr2=0x7ff948197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff94659c640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff948101790 0x7ff948197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff94659c640 1 -- 172.21.15.17:0/1139261541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff948102150 msgr2=0x7ff94819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff94659c640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff948102150 0x7ff94819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff94659c640 1 -- 172.21.15.17:0/1139261541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94819f590 con 0x7ff9481005a0 2024-03-24T02:58:41.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff945d9b640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff948101790 0x7ff948197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:41.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff94659c640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9481005a0 0x7ff948197970 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff94000eb70 tx=0x7ff94000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:41.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff9377fe640 1 -- 172.21.15.17:0/1139261541 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940003b70 con 0x7ff9481005a0 2024-03-24T02:58:41.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff9377fe640 1 -- 172.21.15.17:0/1139261541 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff94000ee70 con 0x7ff9481005a0 2024-03-24T02:58:41.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff94cb77640 1 -- 172.21.15.17:0/1139261541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94819f880 con 0x7ff9481005a0 2024-03-24T02:58:41.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff9377fe640 1 -- 172.21.15.17:0/1139261541 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940010830 con 0x7ff9481005a0 2024-03-24T02:58:41.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.115+0000 7ff94cb77640 1 -- 172.21.15.17:0/1139261541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94819fd80 con 0x7ff9481005a0 2024-03-24T02:58:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.116+0000 7ff94cb77640 1 -- 172.21.15.17:0/1139261541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9481a0040 con 0x7ff9481005a0 2024-03-24T02:58:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.117+0000 7ff9377fe640 1 -- 172.21.15.17:0/1139261541 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff940004a40 con 0x7ff9481005a0 2024-03-24T02:58:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.118+0000 7ff9377fe640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff9300a2800 0x7ff9300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.118+0000 7ff945d9b640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff9300a2800 0x7ff9300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.118+0000 7ff9377fe640 1 -- 172.21.15.17:0/1139261541 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8770+0+0 (secure 0 0 0) 0x7ff9400cf5b0 con 0x7ff9481005a0 2024-03-24T02:58:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.118+0000 7ff945d9b640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff9300a2800 0x7ff9300a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff948198e80 tx=0x7ff93c00b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:41.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.122+0000 7ff9377fe640 1 -- 172.21.15.17:0/1139261541 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff9400203a0 con 0x7ff9481005a0 2024-03-24T02:58:41.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:40 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/464387722' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-24T02:58:41.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:40 smithi188 ceph-mon[36206]: osdmap e87: 12 total, 12 up, 12 in 2024-03-24T02:58:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:40 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/538390550' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-24T02:58:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:40 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/538390550' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-24T02:58:41.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:40 smithi188 ceph-mon[36206]: osdmap e88: 12 total, 12 up, 12 in 2024-03-24T02:58:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:40 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/464387722' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-24T02:58:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:40 smithi019 ceph-mon[35944]: osdmap e87: 12 total, 12 up, 12 in 2024-03-24T02:58:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:40 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/538390550' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-24T02:58:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:40 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/538390550' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-24T02:58:41.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:40 smithi019 ceph-mon[35944]: osdmap e88: 12 total, 12 up, 12 in 2024-03-24T02:58:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.346+0000 7ff94cb77640 1 -- 172.21.15.17:0/1139261541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7ff9481a0210 con 0x7ff9481005a0 2024-03-24T02:58:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:40 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/464387722' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-24T02:58:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:40 smithi017 ceph-mon[31926]: osdmap e87: 12 total, 12 up, 12 in 2024-03-24T02:58:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:40 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/538390550' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-24T02:58:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:40 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/538390550' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-24T02:58:41.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:40 smithi017 ceph-mon[31926]: osdmap e88: 12 total, 12 up, 12 in 2024-03-24T02:58:41.990 INFO:teuthology.orchestra.run.smithi017.stderr:added data pool 4 to fsmap 2024-03-24T02:58:41.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.991+0000 7ff9377fe640 1 -- 172.21.15.17:0/1139261541 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7ff94009c3d0 con 0x7ff9481005a0 2024-03-24T02:58:41.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.993+0000 7ff94cb77640 1 -- 172.21.15.17:0/1139261541 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff9300a2800 msgr2=0x7ff9300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:41.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.993+0000 7ff94cb77640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff9300a2800 0x7ff9300a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff948198e80 tx=0x7ff93c00b5f0 comp rx=0 tx=0).stop 2024-03-24T02:58:41.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.993+0000 7ff94cb77640 1 -- 172.21.15.17:0/1139261541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9481005a0 msgr2=0x7ff948197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:41.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.993+0000 7ff94cb77640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9481005a0 0x7ff948197970 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff94000eb70 tx=0x7ff94000c6a0 comp rx=0 tx=0).stop 2024-03-24T02:58:41.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.994+0000 7ff94cb77640 1 -- 172.21.15.17:0/1139261541 shutdown_connections 2024-03-24T02:58:41.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.994+0000 7ff94cb77640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff9300a2800 0x7ff9300a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:41.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.994+0000 7ff94cb77640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff948102150 0x7ff94819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:41.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.994+0000 7ff94cb77640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff948101790 0x7ff948197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:41.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.994+0000 7ff94cb77640 1 --2- 172.21.15.17:0/1139261541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9481005a0 0x7ff948197970 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:41.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.994+0000 7ff94cb77640 1 -- 172.21.15.17:0/1139261541 >> 172.21.15.17:0/1139261541 conn(0x7ff9480fbb30 msgr2=0x7ff9480fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:41.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.994+0000 7ff94cb77640 1 -- 172.21.15.17:0/1139261541 shutdown_connections 2024-03-24T02:58:41.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:41.994+0000 7ff94cb77640 1 -- 172.21.15.17:0/1139261541 wait complete. 2024-03-24T02:58:42.005 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:42.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.110+0000 7f54ce48a640 1 -- 172.21.15.17:0/3310767040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54c81021f0 msgr2=0x7f54c8108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.110+0000 7f54ce48a640 1 --2- 172.21.15.17:0/3310767040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54c81021f0 0x7f54c8108a80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009ca0 tx=0x7f54c002c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.110+0000 7f54ce48a640 1 -- 172.21.15.17:0/3310767040 shutdown_connections 2024-03-24T02:58:42.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.110+0000 7f54ce48a640 1 --2- 172.21.15.17:0/3310767040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54c81021f0 0x7f54c8108a80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.110+0000 7f54ce48a640 1 --2- 172.21.15.17:0/3310767040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c8101830 0x7f54c8101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.110+0000 7f54ce48a640 1 --2- 172.21.15.17:0/3310767040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c8100640 0x7f54c8100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.110+0000 7f54ce48a640 1 -- 172.21.15.17:0/3310767040 >> 172.21.15.17:0/3310767040 conn(0x7f54c80fbbd0 msgr2=0x7f54c80fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:42.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.110+0000 7f54ce48a640 1 -- 172.21.15.17:0/3310767040 shutdown_connections 2024-03-24T02:58:42.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.110+0000 7f54ce48a640 1 -- 172.21.15.17:0/3310767040 wait complete. 2024-03-24T02:58:42.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.110+0000 7f54ce48a640 1 Processor -- start 2024-03-24T02:58:42.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.110+0000 7f54ce48a640 1 -- start start 2024-03-24T02:58:42.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54ce48a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54c8100640 0x7f54c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:42.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54ce48a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c8101830 0x7f54c819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:42.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54ce48a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c81021f0 0x7f54c819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:42.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54ce48a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c810c250 con 0x7f54c8100640 2024-03-24T02:58:42.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54ce48a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c810c0d0 con 0x7f54c81021f0 2024-03-24T02:58:42.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54ce48a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c810c3d0 con 0x7f54c8101830 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54c7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54c8100640 0x7f54c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54c7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54c8100640 0x7f54c806a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38856/0 (socket says 172.21.15.17:38856) 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54c7fff640 1 -- 172.21.15.17:0/3565333782 learned_addr learned my addr 172.21.15.17:0/3565333782 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54cca27640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c81021f0 0x7f54c819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54c77fe640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c8101830 0x7f54c819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54c7fff640 1 -- 172.21.15.17:0/3565333782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c8101830 msgr2=0x7f54c819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54c7fff640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c8101830 0x7f54c819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54c7fff640 1 -- 172.21.15.17:0/3565333782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c81021f0 msgr2=0x7f54c819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54c7fff640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c81021f0 0x7f54c819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54c7fff640 1 -- 172.21.15.17:0/3565333782 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c819f5e0 con 0x7f54c8100640 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54cca27640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c81021f0 0x7f54c819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54c7fff640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54c8100640 0x7f54c806a840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f54b800d770 tx=0x7f54b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.111+0000 7f54c57fa640 1 -- 172.21.15.17:0/3565333782 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8017070 con 0x7f54c8100640 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.112+0000 7f54c57fa640 1 -- 172.21.15.17:0/3565333782 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54b8009c30 con 0x7f54c8100640 2024-03-24T02:58:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.112+0000 7f54ce48a640 1 -- 172.21.15.17:0/3565333782 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c819f870 con 0x7f54c8100640 2024-03-24T02:58:42.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.112+0000 7f54c57fa640 1 -- 172.21.15.17:0/3565333782 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8003dd0 con 0x7f54c8100640 2024-03-24T02:58:42.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.112+0000 7f54ce48a640 1 -- 172.21.15.17:0/3565333782 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c819fe10 con 0x7f54c8100640 2024-03-24T02:58:42.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.112+0000 7f54ce48a640 1 -- 172.21.15.17:0/3565333782 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c8107960 con 0x7f54c8100640 2024-03-24T02:58:42.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.113+0000 7f54c57fa640 1 -- 172.21.15.17:0/3565333782 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f54b8003fb0 con 0x7f54c8100640 2024-03-24T02:58:42.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.114+0000 7f54c57fa640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f54b00a2880 0x7f54b00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:42.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.114+0000 7f54c77fe640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f54b00a2880 0x7f54b00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:42.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.114+0000 7f54c57fa640 1 -- 172.21.15.17:0/3565333782 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f54b80ce5c0 con 0x7f54c8100640 2024-03-24T02:58:42.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.114+0000 7f54c77fe640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f54b00a2880 0x7f54b00a4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f54c806a5b0 tx=0x7f54bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:42.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.119+0000 7f54c57fa640 1 -- 172.21.15.17:0/3565333782 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f54b8009de0 con 0x7f54c8100640 2024-03-24T02:58:42.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:41 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1139261541' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-24T02:58:42.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:41 smithi188 ceph-mon[36206]: pgmap v294: 129 pgs: 2 creating+peering, 103 active+clean, 24 unknown; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-24T02:58:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:41 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1139261541' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-24T02:58:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:41 smithi019 ceph-mon[35944]: pgmap v294: 129 pgs: 2 creating+peering, 103 active+clean, 24 unknown; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-24T02:58:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.340+0000 7f54ce48a640 1 -- 172.21.15.17:0/3565333782 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54c802d070 con 0x7f54c8100640 2024-03-24T02:58:42.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.341+0000 7f54c57fa640 1 -- 172.21.15.17:0/3565333782 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11318 (secure 0 0 0) 0x7f54b809b420 con 0x7f54c8100640 2024-03-24T02:58:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:42.340 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/1592961709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":1592961709},{"type":"v1","addr":"172.21.15.17:6841","nonce":1592961709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2102503456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2102503456},{"type":"v1","addr":"172.21.15.19:6839","nonce":2102503456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-24T02:58:36.476051+0000","modified":"2024-03-24T02:58:41.347130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24529,"qdb_cluster":[24529]},"id":1}]} 2024-03-24T02:58:42.340 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 17 2024-03-24T02:58:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.343+0000 7f54ce48a640 1 -- 172.21.15.17:0/3565333782 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f54b00a2880 msgr2=0x7f54b00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.343+0000 7f54ce48a640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f54b00a2880 0x7f54b00a4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f54c806a5b0 tx=0x7f54bc005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.343+0000 7f54ce48a640 1 -- 172.21.15.17:0/3565333782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54c8100640 msgr2=0x7f54c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.343+0000 7f54ce48a640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54c8100640 0x7f54c806a840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f54b800d770 tx=0x7f54b800dc30 comp rx=0 tx=0).stop 2024-03-24T02:58:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.343+0000 7f54ce48a640 1 -- 172.21.15.17:0/3565333782 shutdown_connections 2024-03-24T02:58:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.343+0000 7f54ce48a640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f54b00a2880 0x7f54b00a4d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.343+0000 7f54ce48a640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c81021f0 0x7f54c819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.343+0000 7f54ce48a640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c8101830 0x7f54c819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.343+0000 7f54ce48a640 1 --2- 172.21.15.17:0/3565333782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54c8100640 0x7f54c806a840 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.343+0000 7f54ce48a640 1 -- 172.21.15.17:0/3565333782 >> 172.21.15.17:0/3565333782 conn(0x7f54c80fbbd0 msgr2=0x7f54c8104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.343+0000 7f54ce48a640 1 -- 172.21.15.17:0/3565333782 shutdown_connections 2024-03-24T02:58:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.343+0000 7f54ce48a640 1 -- 172.21.15.17:0/3565333782 wait complete. 2024-03-24T02:58:42.352 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:41 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1139261541' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-24T02:58:42.353 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:41 smithi017 ceph-mon[31926]: pgmap v294: 129 pgs: 2 creating+peering, 103 active+clean, 24 unknown; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-24T02:58:42.354 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T02:58:42.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.457+0000 7f0a5d87e640 1 -- 172.21.15.17:0/1533849227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a58102170 msgr2=0x7f0a58108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.457+0000 7f0a5d87e640 1 --2- 172.21.15.17:0/1533849227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a58102170 0x7f0a58108a00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009ca0 tx=0x7f0a5002c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.457+0000 7f0a5d87e640 1 -- 172.21.15.17:0/1533849227 shutdown_connections 2024-03-24T02:58:42.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.457+0000 7f0a5d87e640 1 --2- 172.21.15.17:0/1533849227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a58102170 0x7f0a58108a00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.457+0000 7f0a5d87e640 1 --2- 172.21.15.17:0/1533849227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a581017b0 0x7f0a58101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.457+0000 7f0a5d87e640 1 --2- 172.21.15.17:0/1533849227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a581005c0 0x7f0a581009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.457+0000 7f0a5d87e640 1 -- 172.21.15.17:0/1533849227 >> 172.21.15.17:0/1533849227 conn(0x7f0a580fbb30 msgr2=0x7f0a580fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.457+0000 7f0a5d87e640 1 -- 172.21.15.17:0/1533849227 shutdown_connections 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.457+0000 7f0a5d87e640 1 -- 172.21.15.17:0/1533849227 wait complete. 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a5d87e640 1 Processor -- start 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a5d87e640 1 -- start start 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a5d87e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a581005c0 0x7f0a581979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a5d87e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a581017b0 0x7f0a58197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a5d87e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a58102170 0x7f0a5819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a5d87e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5810c240 con 0x7f0a581005c0 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a5d87e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5810c0c0 con 0x7f0a581017b0 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a5d87e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5810c3c0 con 0x7f0a58102170 2024-03-24T02:58:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a56ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a581005c0 0x7f0a581979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a56ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a581005c0 0x7f0a581979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38868/0 (socket says 172.21.15.17:38868) 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a56ffd640 1 -- 172.21.15.17:0/3886697239 learned_addr learned my addr 172.21.15.17:0/3886697239 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a577fe640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a58102170 0x7f0a5819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a567fc640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a581017b0 0x7f0a58197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a56ffd640 1 -- 172.21.15.17:0/3886697239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a58102170 msgr2=0x7f0a5819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a56ffd640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a58102170 0x7f0a5819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a56ffd640 1 -- 172.21.15.17:0/3886697239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a581017b0 msgr2=0x7f0a58197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a56ffd640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a581017b0 0x7f0a58197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.458+0000 7f0a56ffd640 1 -- 172.21.15.17:0/3886697239 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5819f5c0 con 0x7f0a581005c0 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.459+0000 7f0a56ffd640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a581005c0 0x7f0a581979a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800d950 tx=0x7f0a4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.459+0000 7f0a5c87c640 1 -- 172.21.15.17:0/3886697239 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48017020 con 0x7f0a581005c0 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.459+0000 7f0a5c87c640 1 -- 172.21.15.17:0/3886697239 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a48009c30 con 0x7f0a581005c0 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.459+0000 7f0a5d87e640 1 -- 172.21.15.17:0/3886697239 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5819f8b0 con 0x7f0a581005c0 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.459+0000 7f0a5c87c640 1 -- 172.21.15.17:0/3886697239 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800f740 con 0x7f0a581005c0 2024-03-24T02:58:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.459+0000 7f0a5d87e640 1 -- 172.21.15.17:0/3886697239 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5819fe00 con 0x7f0a581005c0 2024-03-24T02:58:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.460+0000 7f0a5d87e640 1 -- 172.21.15.17:0/3886697239 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a581a0090 con 0x7f0a581005c0 2024-03-24T02:58:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.461+0000 7f0a5c87c640 1 -- 172.21.15.17:0/3886697239 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0a4801a020 con 0x7f0a581005c0 2024-03-24T02:58:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.461+0000 7f0a5c87c640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f0a400a2800 0x7f0a400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.462+0000 7f0a567fc640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f0a400a2800 0x7f0a400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:42.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.462+0000 7f0a5c87c640 1 -- 172.21.15.17:0/3886697239 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f0a480ce960 con 0x7f0a581005c0 2024-03-24T02:58:42.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.462+0000 7f0a567fc640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f0a400a2800 0x7f0a400a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0a58198eb0 tx=0x7f0a4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:42.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.465+0000 7f0a5c87c640 1 -- 172.21.15.17:0/3886697239 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0a48021070 con 0x7f0a581005c0 2024-03-24T02:58:42.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.651+0000 7f0a5d87e640 1 -- 172.21.15.17:0/3886697239 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a581a0760 con 0x7f0a581005c0 2024-03-24T02:58:42.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.653+0000 7f0a5c87c640 1 -- 172.21.15.17:0/3886697239 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24100 (secure 0 0 0) 0x7f0a48009de0 con 0x7f0a581005c0 2024-03-24T02:58:42.652 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:42.652 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:58:41.346962+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:58:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.654+0000 7f0a5d87e640 1 -- 172.21.15.17:0/3886697239 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f0a400a2800 msgr2=0x7f0a400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.654+0000 7f0a5d87e640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f0a400a2800 0x7f0a400a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0a58198eb0 tx=0x7f0a4c005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.654+0000 7f0a5d87e640 1 -- 172.21.15.17:0/3886697239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a581005c0 msgr2=0x7f0a581979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.654+0000 7f0a5d87e640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a581005c0 0x7f0a581979a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800d950 tx=0x7f0a4800de10 comp rx=0 tx=0).stop 2024-03-24T02:58:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.654+0000 7f0a5d87e640 1 -- 172.21.15.17:0/3886697239 shutdown_connections 2024-03-24T02:58:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.654+0000 7f0a5d87e640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f0a400a2800 0x7f0a400a4cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.654+0000 7f0a5d87e640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a58102170 0x7f0a5819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.654+0000 7f0a5d87e640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a581017b0 0x7f0a58197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.654+0000 7f0a5d87e640 1 --2- 172.21.15.17:0/3886697239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a581005c0 0x7f0a581979a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.654+0000 7f0a5d87e640 1 -- 172.21.15.17:0/3886697239 >> 172.21.15.17:0/3886697239 conn(0x7f0a580fbb30 msgr2=0x7f0a580fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:42.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.655+0000 7f0a5d87e640 1 -- 172.21.15.17:0/3886697239 shutdown_connections 2024-03-24T02:58:42.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.655+0000 7f0a5d87e640 1 -- 172.21.15.17:0/3886697239 wait complete. 2024-03-24T02:58:42.665 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T02:58:42.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.768+0000 7f28ccf90640 1 -- 172.21.15.17:0/4013673131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8100640 msgr2=0x7f28c8100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.768+0000 7f28ccf90640 1 --2- 172.21.15.17:0/4013673131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8100640 0x7f28c8100a60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009900 tx=0x7f28b802bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:42.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.768+0000 7f28ccf90640 1 -- 172.21.15.17:0/4013673131 shutdown_connections 2024-03-24T02:58:42.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.768+0000 7f28ccf90640 1 --2- 172.21.15.17:0/4013673131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c81021f0 0x7f28c8108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.768+0000 7f28ccf90640 1 --2- 172.21.15.17:0/4013673131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c8101830 0x7f28c8101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.768+0000 7f28ccf90640 1 --2- 172.21.15.17:0/4013673131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8100640 0x7f28c8100a60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.768+0000 7f28ccf90640 1 -- 172.21.15.17:0/4013673131 >> 172.21.15.17:0/4013673131 conn(0x7f28c80fbbd0 msgr2=0x7f28c80fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:42.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.768+0000 7f28ccf90640 1 -- 172.21.15.17:0/4013673131 shutdown_connections 2024-03-24T02:58:42.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.768+0000 7f28ccf90640 1 -- 172.21.15.17:0/4013673131 wait complete. 2024-03-24T02:58:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28ccf90640 1 Processor -- start 2024-03-24T02:58:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28ccf90640 1 -- start start 2024-03-24T02:58:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28ccf90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c8100640 0x7f28c81979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28ccf90640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8101830 0x7f28c8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28ccf90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c81021f0 0x7f28c819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28ccf90640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c810c220 con 0x7f28c8101830 2024-03-24T02:58:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28ccf90640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c810c0a0 con 0x7f28c81021f0 2024-03-24T02:58:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28ccf90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c810c3a0 con 0x7f28c8100640 2024-03-24T02:58:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28c5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8101830 0x7f28c8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28c5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8101830 0x7f28c8197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38886/0 (socket says 172.21.15.17:38886) 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28c659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c8100640 0x7f28c81979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28c5d9b640 1 -- 172.21.15.17:0/2245334932 learned_addr learned my addr 172.21.15.17:0/2245334932 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28c6d9d640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c81021f0 0x7f28c819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28c5d9b640 1 -- 172.21.15.17:0/2245334932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c8100640 msgr2=0x7f28c81979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28c5d9b640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c8100640 0x7f28c81979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28c5d9b640 1 -- 172.21.15.17:0/2245334932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c81021f0 msgr2=0x7f28c819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.769+0000 7f28c5d9b640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c81021f0 0x7f28c819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.770+0000 7f28c5d9b640 1 -- 172.21.15.17:0/2245334932 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c819f5e0 con 0x7f28c8101830 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.770+0000 7f28c5d9b640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8101830 0x7f28c8197f00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00c890 tx=0x7f28bc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.770+0000 7f28b77fe640 1 -- 172.21.15.17:0/2245334932 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc004220 con 0x7f28c8101830 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.770+0000 7f28b77fe640 1 -- 172.21.15.17:0/2245334932 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28bc0043c0 con 0x7f28c8101830 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.770+0000 7f28ccf90640 1 -- 172.21.15.17:0/2245334932 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c819f8d0 con 0x7f28c8101830 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.770+0000 7f28b77fe640 1 -- 172.21.15.17:0/2245334932 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0049e0 con 0x7f28c8101830 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.770+0000 7f28ccf90640 1 -- 172.21.15.17:0/2245334932 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c819fe20 con 0x7f28c8101830 2024-03-24T02:58:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.771+0000 7f28ccf90640 1 -- 172.21.15.17:0/2245334932 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c81a00b0 con 0x7f28c8101830 2024-03-24T02:58:42.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.772+0000 7f28b77fe640 1 -- 172.21.15.17:0/2245334932 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f28bc0079b0 con 0x7f28c8101830 2024-03-24T02:58:42.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.772+0000 7f28b77fe640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f28b00a2800 0x7f28b00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:42.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.773+0000 7f28c659c640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f28b00a2800 0x7f28b00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:42.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.773+0000 7f28b77fe640 1 -- 172.21.15.17:0/2245334932 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f28bc0cecb0 con 0x7f28c8101830 2024-03-24T02:58:42.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.773+0000 7f28c659c640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f28b00a2800 0x7f28b00a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f28b8000c00 tx=0x7f28b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:42.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.776+0000 7f28b77fe640 1 -- 172.21.15.17:0/2245334932 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f28bc014030 con 0x7f28c8101830 2024-03-24T02:58:42.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.959+0000 7f28ccf90640 1 -- 172.21.15.17:0/2245334932 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28c81a0280 con 0x7f28c8101830 2024-03-24T02:58:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.960+0000 7f28b77fe640 1 -- 172.21.15.17:0/2245334932 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24100 (secure 0 0 0) 0x7f28bc09b460 con 0x7f28c8101830 2024-03-24T02:58:42.960 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:42.960 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:58:41.346962+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:58:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.962+0000 7f28ccf90640 1 -- 172.21.15.17:0/2245334932 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f28b00a2800 msgr2=0x7f28b00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.962+0000 7f28ccf90640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f28b00a2800 0x7f28b00a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f28b8000c00 tx=0x7f28b8005fd0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.962+0000 7f28ccf90640 1 -- 172.21.15.17:0/2245334932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8101830 msgr2=0x7f28c8197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.962+0000 7f28ccf90640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8101830 0x7f28c8197f00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00c890 tx=0x7f28bc00cd50 comp rx=0 tx=0).stop 2024-03-24T02:58:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.962+0000 7f28ccf90640 1 -- 172.21.15.17:0/2245334932 shutdown_connections 2024-03-24T02:58:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.962+0000 7f28ccf90640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f28b00a2800 0x7f28b00a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.962+0000 7f28ccf90640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c81021f0 0x7f28c819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.962+0000 7f28ccf90640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8101830 0x7f28c8197f00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.962+0000 7f28ccf90640 1 --2- 172.21.15.17:0/2245334932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c8100640 0x7f28c81979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.962+0000 7f28ccf90640 1 -- 172.21.15.17:0/2245334932 >> 172.21.15.17:0/2245334932 conn(0x7f28c80fbbd0 msgr2=0x7f28c8069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.962+0000 7f28ccf90640 1 -- 172.21.15.17:0/2245334932 shutdown_connections 2024-03-24T02:58:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:42.962+0000 7f28ccf90640 1 -- 172.21.15.17:0/2245334932 wait complete. 2024-03-24T02:58:42.987 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:43.047 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:58:42 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i[67156]: 2024-03-24T02:58:42.748+0000 7f913f57f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-24T02:58:43.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:42 smithi188 ceph-mon[36206]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T02:58:43.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:42 smithi188 ceph-mon[36206]: osdmap e89: 12 total, 12 up, 12 in 2024-03-24T02:58:43.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:42 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1139261541' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-24T02:58:43.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:42 smithi188 ceph-mon[36206]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:43.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:42 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3565333782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:43.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:42 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3886697239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:43.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:42 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2245334932' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.091+0000 7fdaba659640 1 -- 172.21.15.17:0/2892538371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdab41016c0 msgr2=0x7fdab4101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.091+0000 7fdaba659640 1 --2- 172.21.15.17:0/2892538371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdab41016c0 0x7fdab4101ae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009900 tx=0x7fdaa402bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.091+0000 7fdaba659640 1 -- 172.21.15.17:0/2892538371 shutdown_connections 2024-03-24T02:58:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.091+0000 7fdaba659640 1 --2- 172.21.15.17:0/2892538371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdab4103270 0x7fdab4109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.091+0000 7fdaba659640 1 --2- 172.21.15.17:0/2892538371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab41028b0 0x7fdab4102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.091+0000 7fdaba659640 1 --2- 172.21.15.17:0/2892538371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdab41016c0 0x7fdab4101ae0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.091+0000 7fdaba659640 1 -- 172.21.15.17:0/2892538371 >> 172.21.15.17:0/2892538371 conn(0x7fdab40fcc50 msgr2=0x7fdab40ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.091+0000 7fdaba659640 1 -- 172.21.15.17:0/2892538371 shutdown_connections 2024-03-24T02:58:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.091+0000 7fdaba659640 1 -- 172.21.15.17:0/2892538371 wait complete. 2024-03-24T02:58:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdaba659640 1 Processor -- start 2024-03-24T02:58:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdaba659640 1 -- start start 2024-03-24T02:58:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdaba659640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdab41016c0 0x7fdab41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdaba659640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab41028b0 0x7fdab4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdaba659640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdab4103270 0x7fdab419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdaba659640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab410d340 con 0x7fdab4103270 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdaba659640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab410d1c0 con 0x7fdab41028b0 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdaba659640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab410d4c0 con 0x7fdab41016c0 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdab8bf6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdab4103270 0x7fdab419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdab8bf6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdab4103270 0x7fdab419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38906/0 (socket says 172.21.15.17:38906) 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdab8bf6640 1 -- 172.21.15.17:0/579264204 learned_addr learned my addr 172.21.15.17:0/579264204 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdab3fff640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdab41016c0 0x7fdab41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdab37fe640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab41028b0 0x7fdab4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdab8bf6640 1 -- 172.21.15.17:0/579264204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdab41016c0 msgr2=0x7fdab41979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdab8bf6640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdab41016c0 0x7fdab41979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdab8bf6640 1 -- 172.21.15.17:0/579264204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab41028b0 msgr2=0x7fdab4197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdab8bf6640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab41028b0 0x7fdab4197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdab8bf6640 1 -- 172.21.15.17:0/579264204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab419f5d0 con 0x7fdab4103270 2024-03-24T02:58:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.092+0000 7fdab8bf6640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdab4103270 0x7fdab419ef60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00eb10 tx=0x7fdaac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.093+0000 7fdab17fa640 1 -- 172.21.15.17:0/579264204 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac004b20 con 0x7fdab4103270 2024-03-24T02:58:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.093+0000 7fdaba659640 1 -- 172.21.15.17:0/579264204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab419f8c0 con 0x7fdab4103270 2024-03-24T02:58:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.093+0000 7fdab17fa640 1 -- 172.21.15.17:0/579264204 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdaac004cc0 con 0x7fdab4103270 2024-03-24T02:58:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.093+0000 7fdab17fa640 1 -- 172.21.15.17:0/579264204 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac010760 con 0x7fdab4103270 2024-03-24T02:58:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.093+0000 7fdaba659640 1 -- 172.21.15.17:0/579264204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab419fe10 con 0x7fdab4103270 2024-03-24T02:58:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.094+0000 7fdaba659640 1 -- 172.21.15.17:0/579264204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab41089e0 con 0x7fdab4103270 2024-03-24T02:58:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.095+0000 7fdab17fa640 1 -- 172.21.15.17:0/579264204 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdaac005000 con 0x7fdab4103270 2024-03-24T02:58:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.095+0000 7fdab17fa640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda9c0a2800 0x7fda9c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.095+0000 7fdab17fa640 1 -- 172.21.15.17:0/579264204 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdaac0cf780 con 0x7fdab4103270 2024-03-24T02:58:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.095+0000 7fdab3fff640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda9c0a2800 0x7fda9c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.096+0000 7fdab3fff640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda9c0a2800 0x7fda9c0a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4000c00 tx=0x7fdaa4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.099+0000 7fdab17fa640 1 -- 172.21.15.17:0/579264204 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdaac09c5e0 con 0x7fdab4103270 2024-03-24T02:58:43.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.320+0000 7fdaba659640 1 -- 172.21.15.17:0/579264204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdab402d2c0 con 0x7fdab4103270 2024-03-24T02:58:43.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:42 smithi019 ceph-mon[35944]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T02:58:43.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:42 smithi019 ceph-mon[35944]: osdmap e89: 12 total, 12 up, 12 in 2024-03-24T02:58:43.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:42 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1139261541' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-24T02:58:43.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:42 smithi019 ceph-mon[35944]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:43.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:42 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3565333782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:43.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:42 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3886697239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:43.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:42 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2245334932' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:43.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.321+0000 7fdab17fa640 1 -- 172.21.15.17:0/579264204 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11318 (secure 0 0 0) 0x7fdaac09bee0 con 0x7fdab4103270 2024-03-24T02:58:43.321 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:43.321 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/1592961709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":1592961709},{"type":"v1","addr":"172.21.15.17:6841","nonce":1592961709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2102503456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2102503456},{"type":"v1","addr":"172.21.15.19:6839","nonce":2102503456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-24T02:58:36.476051+0000","modified":"2024-03-24T02:58:41.347130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24529,"qdb_cluster":[24529]},"id":1}]} 2024-03-24T02:58:43.322 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 17 2024-03-24T02:58:43.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.323+0000 7fdaba659640 1 -- 172.21.15.17:0/579264204 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda9c0a2800 msgr2=0x7fda9c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.323+0000 7fdaba659640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda9c0a2800 0x7fda9c0a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4000c00 tx=0x7fdaa4005fd0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.323+0000 7fdaba659640 1 -- 172.21.15.17:0/579264204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdab4103270 msgr2=0x7fdab419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.323+0000 7fdaba659640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdab4103270 0x7fdab419ef60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00eb10 tx=0x7fdaac00c2d0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.323+0000 7fdaba659640 1 -- 172.21.15.17:0/579264204 shutdown_connections 2024-03-24T02:58:43.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.323+0000 7fdaba659640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda9c0a2800 0x7fda9c0a4cc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.323+0000 7fdaba659640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdab4103270 0x7fdab419ef60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.323+0000 7fdaba659640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab41028b0 0x7fdab4197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.323+0000 7fdaba659640 1 --2- 172.21.15.17:0/579264204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdab41016c0 0x7fdab41979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.323+0000 7fdaba659640 1 -- 172.21.15.17:0/579264204 >> 172.21.15.17:0/579264204 conn(0x7fdab40fcc50 msgr2=0x7fdab4105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:43.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.323+0000 7fdaba659640 1 -- 172.21.15.17:0/579264204 shutdown_connections 2024-03-24T02:58:43.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.323+0000 7fdaba659640 1 -- 172.21.15.17:0/579264204 wait complete. 2024-03-24T02:58:43.332 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:42 smithi017 ceph-mon[31926]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T02:58:43.332 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:42 smithi017 ceph-mon[31926]: osdmap e89: 12 total, 12 up, 12 in 2024-03-24T02:58:43.332 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:42 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1139261541' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-24T02:58:43.332 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:42 smithi017 ceph-mon[31926]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:43.332 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:42 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3565333782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:43.333 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:42 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3886697239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:43.333 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:42 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2245334932' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:43.334 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T02:58:43.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.437+0000 7f6f2b061640 1 -- 172.21.15.17:0/1327987621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f24102190 msgr2=0x7f6f24108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.437+0000 7f6f2b061640 1 --2- 172.21.15.17:0/1327987621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f24102190 0x7f6f24108a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009ca0 tx=0x7f6f1c02c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.437+0000 7f6f2b061640 1 -- 172.21.15.17:0/1327987621 shutdown_connections 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.437+0000 7f6f2b061640 1 --2- 172.21.15.17:0/1327987621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f24102190 0x7f6f24108a20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.437+0000 7f6f2b061640 1 --2- 172.21.15.17:0/1327987621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f241017d0 0x7f6f24101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.437+0000 7f6f2b061640 1 --2- 172.21.15.17:0/1327987621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f241005e0 0x7f6f24100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.437+0000 7f6f2b061640 1 -- 172.21.15.17:0/1327987621 >> 172.21.15.17:0/1327987621 conn(0x7f6f240fbb50 msgr2=0x7f6f240fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.437+0000 7f6f2b061640 1 -- 172.21.15.17:0/1327987621 shutdown_connections 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.437+0000 7f6f2b061640 1 -- 172.21.15.17:0/1327987621 wait complete. 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.437+0000 7f6f2b061640 1 Processor -- start 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.437+0000 7f6f2b061640 1 -- start start 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f2b061640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f241005e0 0x7f6f2406a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f2b061640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f241017d0 0x7f6f2419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f2b061640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f24102190 0x7f6f2419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f2b061640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2410c260 con 0x7f6f241017d0 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f2b061640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2410c0e0 con 0x7f6f24102190 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f2b061640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2410c3e0 con 0x7f6f241005e0 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f23fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f241017d0 0x7f6f2419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f23fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f241017d0 0x7f6f2419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38920/0 (socket says 172.21.15.17:38920) 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f23fff640 1 -- 172.21.15.17:0/1413572873 learned_addr learned my addr 172.21.15.17:0/1413572873 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f28dfd640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f241005e0 0x7f6f2406a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f23fff640 1 -- 172.21.15.17:0/1413572873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f241005e0 msgr2=0x7f6f2406a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f23fff640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f241005e0 0x7f6f2406a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f23fff640 1 -- 172.21.15.17:0/1413572873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f24102190 msgr2=0x7f6f2419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f23fff640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f24102190 0x7f6f2419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f23fff640 1 -- 172.21.15.17:0/1413572873 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2419f5d0 con 0x7f6f241017d0 2024-03-24T02:58:43.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f23fff640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f241017d0 0x7f6f2419c6d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00d950 tx=0x7f6f0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:43.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f21ffb640 1 -- 172.21.15.17:0/1413572873 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c017030 con 0x7f6f241017d0 2024-03-24T02:58:43.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f21ffb640 1 -- 172.21.15.17:0/1413572873 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f0c004020 con 0x7f6f241017d0 2024-03-24T02:58:43.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f2b061640 1 -- 172.21.15.17:0/1413572873 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2419f8c0 con 0x7f6f241017d0 2024-03-24T02:58:43.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.438+0000 7f6f21ffb640 1 -- 172.21.15.17:0/1413572873 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c0047a0 con 0x7f6f241017d0 2024-03-24T02:58:43.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.439+0000 7f6f2b061640 1 -- 172.21.15.17:0/1413572873 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2419fe60 con 0x7f6f241017d0 2024-03-24T02:58:43.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.440+0000 7f6f2b061640 1 -- 172.21.15.17:0/1413572873 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f241078a0 con 0x7f6f241017d0 2024-03-24T02:58:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.440+0000 7f6f21ffb640 1 -- 172.21.15.17:0/1413572873 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6f0c0041d0 con 0x7f6f241017d0 2024-03-24T02:58:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.441+0000 7f6f21ffb640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6f100a2880 0x7f6f100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.441+0000 7f6f28dfd640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6f100a2880 0x7f6f100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.441+0000 7f6f21ffb640 1 -- 172.21.15.17:0/1413572873 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f6f0c0ce5f0 con 0x7f6f241017d0 2024-03-24T02:58:43.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.442+0000 7f6f28dfd640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6f100a2880 0x7f6f100a4d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6f24101630 tx=0x7f6f14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:43.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.446+0000 7f6f21ffb640 1 -- 172.21.15.17:0/1413572873 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6f0c09b450 con 0x7f6f241017d0 2024-03-24T02:58:43.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.632+0000 7f6f2b061640 1 -- 172.21.15.17:0/1413572873 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f2402d070 con 0x7f6f241017d0 2024-03-24T02:58:43.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.633+0000 7f6f21ffb640 1 -- 172.21.15.17:0/1413572873 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24100 (secure 0 0 0) 0x7f6f0c09ad50 con 0x7f6f241017d0 2024-03-24T02:58:43.633 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:43.633 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:58:41.346962+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:58:43.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.635+0000 7f6f2b061640 1 -- 172.21.15.17:0/1413572873 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6f100a2880 msgr2=0x7f6f100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.635+0000 7f6f2b061640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6f100a2880 0x7f6f100a4d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6f24101630 tx=0x7f6f14005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:43.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.635+0000 7f6f2b061640 1 -- 172.21.15.17:0/1413572873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f241017d0 msgr2=0x7f6f2419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.635+0000 7f6f2b061640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f241017d0 0x7f6f2419c6d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00d950 tx=0x7f6f0c00de10 comp rx=0 tx=0).stop 2024-03-24T02:58:43.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.635+0000 7f6f2b061640 1 -- 172.21.15.17:0/1413572873 shutdown_connections 2024-03-24T02:58:43.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.635+0000 7f6f2b061640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6f100a2880 0x7f6f100a4d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.635+0000 7f6f2b061640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f24102190 0x7f6f2419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.635+0000 7f6f2b061640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f241017d0 0x7f6f2419c6d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.635+0000 7f6f2b061640 1 --2- 172.21.15.17:0/1413572873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f241005e0 0x7f6f2406a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.636+0000 7f6f2b061640 1 -- 172.21.15.17:0/1413572873 >> 172.21.15.17:0/1413572873 conn(0x7f6f240fbb50 msgr2=0x7f6f24104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:43.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.636+0000 7f6f2b061640 1 -- 172.21.15.17:0/1413572873 shutdown_connections 2024-03-24T02:58:43.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.636+0000 7f6f2b061640 1 -- 172.21.15.17:0/1413572873 wait complete. 2024-03-24T02:58:43.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-24T02:58:36.476051+0000', 'modified': '2024-03-24T02:58:41.347130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24529}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.188:6839/81409664', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 81409664}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 81409664}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24529, 'qdb_cluster': [24529]} 2024-03-24T02:58:43.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-24T02:58:43.647 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-24T02:58:43.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 -- 172.21.15.17:0/3210283276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8dc1005b0 msgr2=0x7fb8dc1009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 --2- 172.21.15.17:0/3210283276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8dc1005b0 0x7fb8dc1009d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009860 tx=0x7fb8d402bb70 comp rx=0 tx=0).stop 2024-03-24T02:58:43.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 -- 172.21.15.17:0/3210283276 shutdown_connections 2024-03-24T02:58:43.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 --2- 172.21.15.17:0/3210283276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8dc102160 0x7fb8dc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 --2- 172.21.15.17:0/3210283276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8dc1017a0 0x7fb8dc101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 --2- 172.21.15.17:0/3210283276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8dc1005b0 0x7fb8dc1009d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 -- 172.21.15.17:0/3210283276 >> 172.21.15.17:0/3210283276 conn(0x7fb8dc0fbb40 msgr2=0x7fb8dc0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:43.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 -- 172.21.15.17:0/3210283276 shutdown_connections 2024-03-24T02:58:43.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 -- 172.21.15.17:0/3210283276 wait complete. 2024-03-24T02:58:43.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 Processor -- start 2024-03-24T02:58:43.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 -- start start 2024-03-24T02:58:43.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8dc1005b0 0x7fb8dc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8dc1017a0 0x7fb8dc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8dc102160 0x7fb8dc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc10d310 con 0x7fb8dc1017a0 2024-03-24T02:58:43.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc10d190 con 0x7fb8dc102160 2024-03-24T02:58:43.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.752+0000 7fb8e4668640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc10d490 con 0x7fb8dc1005b0 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e1c03640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8dc1017a0 0x7fb8dc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e1c03640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8dc1017a0 0x7fb8dc197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38942/0 (socket says 172.21.15.17:38942) 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e1c03640 1 -- 172.21.15.17:0/1691461716 learned_addr learned my addr 172.21.15.17:0/1691461716 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e2404640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8dc1005b0 0x7fb8dc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e1c03640 1 -- 172.21.15.17:0/1691461716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8dc1005b0 msgr2=0x7fb8dc197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8dc1005b0 0x7fb8dc197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e1c03640 1 -- 172.21.15.17:0/1691461716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8dc102160 msgr2=0x7fb8dc19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e2c05640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8dc102160 0x7fb8dc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8dc102160 0x7fb8dc19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e1c03640 1 -- 172.21.15.17:0/1691461716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc19f660 con 0x7fb8dc1017a0 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8dc1017a0 0x7fb8dc197f80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00c890 tx=0x7fb8cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8d37fe640 1 -- 172.21.15.17:0/1691461716 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc004220 con 0x7fb8dc1017a0 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8d37fe640 1 -- 172.21.15.17:0/1691461716 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8cc0043c0 con 0x7fb8dc1017a0 2024-03-24T02:58:43.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e4668640 1 -- 172.21.15.17:0/1691461716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc19f950 con 0x7fb8dc1017a0 2024-03-24T02:58:43.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8d37fe640 1 -- 172.21.15.17:0/1691461716 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0049e0 con 0x7fb8dc1017a0 2024-03-24T02:58:43.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.753+0000 7fb8e4668640 1 -- 172.21.15.17:0/1691461716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc19fea0 con 0x7fb8dc1017a0 2024-03-24T02:58:43.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.755+0000 7fb8d37fe640 1 -- 172.21.15.17:0/1691461716 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb8cc0079b0 con 0x7fb8dc1017a0 2024-03-24T02:58:43.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.756+0000 7fb8e724e740 1 Processor -- start 2024-03-24T02:58:43.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.756+0000 7fb8d37fe640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.756+0000 7fb8e724e740 1 -- start start 2024-03-24T02:58:43.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.756+0000 7fb8e724e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce3351740 0x559ce3351b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.756+0000 7fb8e2c05640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.756+0000 7fb8e724e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3356c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.756+0000 7fb8d37fe640 1 -- 172.21.15.17:0/1691461716 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb8cc0ced00 con 0x7fb8dc1017a0 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.756+0000 7fb8e724e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 0x559ce3359730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.756+0000 7fb8e724e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce32b6c20 con 0x559ce33572a0 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.756+0000 7fb8e724e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce325fa40 con 0x559ce3351740 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.756+0000 7fb8e724e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce32b2cc0 con 0x559ce3352130 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e2c05640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 0x559ce3359730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e2c05640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 0x559ce3359730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38944/0 (socket says 172.21.15.17:38944) 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e2c05640 1 -- 172.21.15.17:0/951906826 learned_addr learned my addr 172.21.15.17:0/951906826 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/951906826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3356c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e2404640 1 --2- 172.21.15.17:0/951906826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce3351740 0x559ce3351b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e1c03640 1 -- 172.21.15.17:0/951906826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce3351740 msgr2=0x559ce3351b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/951906826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce3351740 0x559ce3351b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e1c03640 1 -- 172.21.15.17:0/951906826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 msgr2=0x559ce3359730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T02:58:43.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/951906826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 0x559ce3359730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e1c03640 1 -- 172.21.15.17:0/951906826 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ce3281f90 con 0x559ce3352130 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e2c05640 1 --2- 172.21.15.17:0/951906826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 0x559ce3359730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e2c05640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8012ba0 tx=0x7fb8d8009d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/951906826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3356c30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc095fd0 tx=0x7fb8cc0054a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd3851cd04265b8a server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8d0bfe640 1 -- 172.21.15.17:0/951906826 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc01d040 con 0x559ce3352130 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8d0bfe640 1 -- 172.21.15.17:0/951906826 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8cc09b460 con 0x559ce3352130 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.757+0000 7fb8d0bfe640 1 -- 172.21.15.17:0/951906826 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc09f120 con 0x559ce3352130 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.758+0000 7fb8e724e740 1 -- 172.21.15.17:0/951906826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 msgr2=0x559ce3356c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.758+0000 7fb8e724e740 1 --2- 172.21.15.17:0/951906826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3356c30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc095fd0 tx=0x7fb8cc0054a0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.758+0000 7fb8e724e740 1 -- 172.21.15.17:0/951906826 shutdown_connections 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.758+0000 7fb8e724e740 1 --2- 172.21.15.17:0/951906826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 0x559ce3359730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.758+0000 7fb8e724e740 1 --2- 172.21.15.17:0/951906826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3356c30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.758+0000 7fb8e724e740 1 --2- 172.21.15.17:0/951906826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce3351740 0x559ce3351b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.758+0000 7fb8e724e740 1 -- 172.21.15.17:0/951906826 >> 172.21.15.17:0/951906826 conn(0x559ce3319a60 msgr2=0x559ce334d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.758+0000 7fb8e724e740 1 -- 172.21.15.17:0/951906826 shutdown_connections 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.758+0000 7fb8e724e740 1 -- 172.21.15.17:0/951906826 wait complete. 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.758+0000 7fb8e724e740 1 -- start start 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.758+0000 7fb8e724e740 1 Processor -- start 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e724e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 0x559ce335bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e724e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce335c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e724e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce33572a0 0x559ce33630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e724e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce335a1f0 con 0x559ce3351740 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e724e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3272e90 con 0x559ce33572a0 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e724e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce335a370 con 0x559ce3352130 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e724e740 10 client.0 _ll_register_callbacks cb 0x559ce2fcaac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e2404640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 0x559ce335bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e2404640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 0x559ce335bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38948/0 (socket says 172.21.15.17:38948) 2024-03-24T02:58:43.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e2404640 1 -- 172.21.15.17:0/3634756669 learned_addr learned my addr 172.21.15.17:0/3634756669 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/3634756669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce335c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e2404640 1 -- 172.21.15.17:0/3634756669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 msgr2=0x559ce335c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e2404640 1 --2- 172.21.15.17:0/3634756669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce335c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e2404640 1 -- 172.21.15.17:0/3634756669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce33572a0 msgr2=0x559ce33630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e2404640 1 --2- 172.21.15.17:0/3634756669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce33572a0 0x559ce33630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e2404640 1 -- 172.21.15.17:0/3634756669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ce327f9d0 con 0x559ce3351740 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e2404640 1 --2- 172.21.15.17:0/3634756669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 0x559ce335bb40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb8d402c180 tx=0x7fb8d40323f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf98add92342abc server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8c5ffb640 1 -- 172.21.15.17:0/3634756669 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4043070 con 0x559ce3351740 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8c5ffb640 1 -- 172.21.15.17:0/3634756669 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb8d4004240 con 0x559ce3351740 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.759+0000 7fb8e724e740 1 -- 172.21.15.17:0/3634756669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8cc09b460 con 0x559ce3351740 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.760+0000 7fb8c5ffb640 1 -- 172.21.15.17:0/3634756669 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8d404f050 con 0x559ce3351740 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.760+0000 7fb8e724e740 10 client.14769 fetch_fsmap learned FSMap version 17 2024-03-24T02:58:43.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.760+0000 7fb8e724e740 1 -- 172.21.15.17:0/3634756669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x559ce2fcb2d0 con 0x559ce3351740 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.760+0000 7fb8c5ffb640 1 -- 172.21.15.17:0/3634756669 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8cc09b460 con 0x559ce3351740 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.760+0000 7fb8c5ffb640 1 -- 172.21.15.17:0/3634756669 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb8d4032870 con 0x559ce3351740 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.760+0000 7fb8e724e740 10 client.14769 fetch_fsmap finished waiting for FSMap version 17 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.760+0000 7fb8e724e740 10 client.14769 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.760+0000 7fb8e724e740 20 client.14769 populate_metadata read hostname 'smithi017' 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.760+0000 7fb8e724e740 1 --2- 172.21.15.17:0/3634756669 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x559ce33683d0 0x559ce33673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.760+0000 7fb8e724e740 4 client.14769 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.760+0000 7fb8e724e740 1 -- 172.21.15.17:0/3634756669 --> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ce32c4160 con 0x559ce33683d0 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.760+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/3634756669 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x559ce33683d0 0x559ce33673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8c5ffb640 10 client.14769 ms_handle_connect on v2:172.21.15.188:6838/81409664 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/3634756669 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x559ce33683d0 0x559ce33673b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa005935f1c43e2f server_cookie=91c1adec3dfeaceb in_seq=0 out_seq=0 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8c5ffb640 1 -- 172.21.15.17:0/3634756669 <== mds.0 v2:172.21.15.188:6838/81409664 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559ce32c4160 con 0x559ce33683d0 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8c5ffb640 10 client.14769 handle_command_reply: tid=0 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8e724e740 1 client.14769 shutdown 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8e724e740 1 -- 172.21.15.17:0/3634756669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 msgr2=0x559ce335bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8e724e740 1 --2- 172.21.15.17:0/3634756669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 0x559ce335bb40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb8d402c180 tx=0x7fb8d40323f0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8e724e740 1 -- 172.21.15.17:0/3634756669 shutdown_connections 2024-03-24T02:58:43.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8e724e740 1 --2- 172.21.15.17:0/3634756669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce33572a0 0x559ce33630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8e724e740 1 --2- 172.21.15.17:0/3634756669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce335c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8e724e740 1 --2- 172.21.15.17:0/3634756669 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x559ce33683d0 0x559ce33673b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8e724e740 1 --2- 172.21.15.17:0/3634756669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 0x559ce335bb40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8c5ffb640 0 client.14769 ms_handle_reset on v2:172.21.15.188:6838/81409664 2024-03-24T02:58:43.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8e724e740 1 -- 172.21.15.17:0/3634756669 >> 172.21.15.17:0/3634756669 conn(0x559ce3319a60 msgr2=0x559ce334c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:43.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8e724e740 1 -- 172.21.15.17:0/3634756669 shutdown_connections 2024-03-24T02:58:43.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.761+0000 7fb8e724e740 1 -- 172.21.15.17:0/3634756669 wait complete. 2024-03-24T02:58:43.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.762+0000 7fb8e724e740 20 client.14769 trim_cache size 0 max 16384 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.778+0000 7fb8e724e740 1 Processor -- start 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.778+0000 7fb8e724e740 1 -- start start 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.778+0000 7fb8e724e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce3351740 0x559ce3353590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.778+0000 7fb8e724e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3353ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.778+0000 7fb8e724e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 0x559ce3363080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.778+0000 7fb8e724e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3367bf0 con 0x559ce33572a0 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.778+0000 7fb8e724e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3368250 con 0x559ce3351740 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.778+0000 7fb8e724e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce33688b0 con 0x559ce3352130 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.778+0000 7fb8e2c05640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 0x559ce3363080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.778+0000 7fb8e2c05640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 0x559ce3363080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38960/0 (socket says 172.21.15.17:38960) 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.778+0000 7fb8e2c05640 1 -- 172.21.15.17:0/1891752723 learned_addr learned my addr 172.21.15.17:0/1891752723 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/1891752723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3353ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8e2404640 1 --2- 172.21.15.17:0/1891752723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce3351740 0x559ce3353590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8e2c05640 1 -- 172.21.15.17:0/1891752723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 msgr2=0x559ce3353ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8e2c05640 1 --2- 172.21.15.17:0/1891752723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3353ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8e2c05640 1 -- 172.21.15.17:0/1891752723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce3351740 msgr2=0x559ce3353590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8e2c05640 1 --2- 172.21.15.17:0/1891752723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce3351740 0x559ce3353590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8e2c05640 1 -- 172.21.15.17:0/1891752723 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ce30956b0 con 0x559ce33572a0 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8e2404640 1 --2- 172.21.15.17:0/1891752723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce3351740 0x559ce3353590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8e2c05640 1 --2- 172.21.15.17:0/1891752723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 0x559ce3363080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8005e30 tx=0x7fb8d801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91f414ac175afe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8c7fff640 1 -- 172.21.15.17:0/1891752723 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8004b20 con 0x559ce33572a0 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8c7fff640 1 -- 172.21.15.17:0/1891752723 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8d8024070 con 0x559ce33572a0 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8c7fff640 1 -- 172.21.15.17:0/1891752723 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d80054f0 con 0x559ce33572a0 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8e724e740 1 -- 172.21.15.17:0/1891752723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 msgr2=0x559ce3363080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.779+0000 7fb8e724e740 1 --2- 172.21.15.17:0/1891752723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 0x559ce3363080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8005e30 tx=0x7fb8d801c000 comp rx=0 tx=0).stop 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 -- 172.21.15.17:0/1891752723 shutdown_connections 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 --2- 172.21.15.17:0/1891752723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce33572a0 0x559ce3363080 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 --2- 172.21.15.17:0/1891752723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3353ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 --2- 172.21.15.17:0/1891752723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce3351740 0x559ce3353590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 -- 172.21.15.17:0/1891752723 >> 172.21.15.17:0/1891752723 conn(0x559ce3319a60 msgr2=0x559ce33673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 -- 172.21.15.17:0/1891752723 shutdown_connections 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 -- 172.21.15.17:0/1891752723 wait complete. 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 -- start start 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 Processor -- start 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 0x559ce3362900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3362e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce33572a0 0x559ce335f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3369570 con 0x559ce3351740 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3368f10 con 0x559ce33572a0 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3369bd0 con 0x559ce3352130 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e724e740 10 client.0 _ll_register_callbacks cb 0x7fb8d40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e2404640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 0x559ce3362900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e2404640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 0x559ce3362900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38962/0 (socket says 172.21.15.17:38962) 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e2404640 1 -- 172.21.15.17:0/359960049 learned_addr learned my addr 172.21.15.17:0/359960049 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/359960049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3362e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e2404640 1 -- 172.21.15.17:0/359960049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 msgr2=0x559ce3362e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e2404640 1 --2- 172.21.15.17:0/359960049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3362e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e2404640 1 -- 172.21.15.17:0/359960049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce33572a0 msgr2=0x559ce335f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e2404640 1 --2- 172.21.15.17:0/359960049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce33572a0 0x559ce335f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.780+0000 7fb8e2404640 1 -- 172.21.15.17:0/359960049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ce335f9e0 con 0x559ce3351740 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8e2404640 1 --2- 172.21.15.17:0/359960049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 0x559ce3362900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4005c40 tx=0x7fb8d4005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e08587053e773d server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8c5ffb640 1 -- 172.21.15.17:0/359960049 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4043040 con 0x559ce3351740 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8c5ffb640 1 -- 172.21.15.17:0/359960049 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb8d4005040 con 0x559ce3351740 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8e724e740 1 -- 172.21.15.17:0/359960049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8d8024070 con 0x559ce3351740 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8c5ffb640 1 -- 172.21.15.17:0/359960049 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8d4061050 con 0x559ce3351740 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8e724e740 10 client.14775 fetch_fsmap learned FSMap version 17 2024-03-24T02:58:43.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8e724e740 1 -- 172.21.15.17:0/359960049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x559ce33c4020 con 0x559ce3351740 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8c5ffb640 1 -- 172.21.15.17:0/359960049 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8d8024070 con 0x559ce3351740 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8c5ffb640 1 -- 172.21.15.17:0/359960049 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb8d40046c0 con 0x559ce3351740 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8e724e740 10 client.14775 fetch_fsmap finished waiting for FSMap version 17 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8e724e740 10 client.14775 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8e724e740 20 client.14775 populate_metadata read hostname 'smithi017' 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8e724e740 1 --2- 172.21.15.17:0/359960049 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x559ce33c8700 0x559ce33c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8e724e740 4 client.14775 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8e724e740 1 -- 172.21.15.17:0/359960049 --> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559ce32be4b0 con 0x559ce33c8700 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.781+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/359960049 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x559ce33c8700 0x559ce33c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e1c03640 1 --2- 172.21.15.17:0/359960049 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x559ce33c8700 0x559ce33c4250 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18c6fbff7c78f16c server_cookie=72bc1a9b6bff0162 in_seq=0 out_seq=0 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8c5ffb640 10 client.14775 ms_handle_connect on v2:172.21.15.188:6838/81409664 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8c5ffb640 1 -- 172.21.15.17:0/359960049 <== mds.0 v2:172.21.15.188:6838/81409664 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x559ce32be4b0 con 0x559ce33c8700 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8c5ffb640 10 client.14775 handle_command_reply: tid=0 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e724e740 1 client.14775 shutdown 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e724e740 1 -- 172.21.15.17:0/359960049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 msgr2=0x559ce3362900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e724e740 1 --2- 172.21.15.17:0/359960049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 0x559ce3362900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4005c40 tx=0x7fb8d4005fa0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e724e740 1 -- 172.21.15.17:0/359960049 shutdown_connections 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e724e740 1 --2- 172.21.15.17:0/359960049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ce33572a0 0x559ce335f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e724e740 1 --2- 172.21.15.17:0/359960049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559ce3352130 0x559ce3362e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e724e740 1 --2- 172.21.15.17:0/359960049 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x559ce33c8700 0x559ce33c4250 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e724e740 1 --2- 172.21.15.17:0/359960049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ce3351740 0x559ce3362900 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8c5ffb640 0 client.14775 ms_handle_reset on v2:172.21.15.188:6838/81409664 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e724e740 1 -- 172.21.15.17:0/359960049 >> 172.21.15.17:0/359960049 conn(0x559ce3319a60 msgr2=0x559ce3353590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e724e740 1 -- 172.21.15.17:0/359960049 shutdown_connections 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e724e740 1 -- 172.21.15.17:0/359960049 wait complete. 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.782+0000 7fb8e724e740 20 client.14775 trim_cache size 0 max 16384 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T02:58:43.788 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-24T02:58:43.789 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24529, 2024-03-24T02:58:43.789 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-24T02:58:43.789 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-24T02:58:43.789 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-24T02:58:43.789 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 6.4092240900000004, 2024-03-24T02:58:43.789 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 17, 2024-03-24T02:58:43.789 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 89, 2024-03-24T02:58:43.789 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-03-24T02:58:43.789 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 21.382672652 2024-03-24T02:58:43.789 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T02:58:43.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.783+0000 7fb8e4668640 1 -- 172.21.15.17:0/1691461716 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb8c80a2800 msgr2=0x7fb8c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.783+0000 7fb8e4668640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8012ba0 tx=0x7fb8d8009d70 comp rx=0 tx=0).stop 2024-03-24T02:58:43.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.783+0000 7fb8e4668640 1 -- 172.21.15.17:0/1691461716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8dc1017a0 msgr2=0x7fb8dc197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:43.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.783+0000 7fb8e4668640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8dc1017a0 0x7fb8dc197f80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00c890 tx=0x7fb8cc00cd50 comp rx=0 tx=0).stop 2024-03-24T02:58:43.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.783+0000 7fb8e4668640 1 -- 172.21.15.17:0/1691461716 shutdown_connections 2024-03-24T02:58:43.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.783+0000 7fb8e4668640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.783+0000 7fb8e4668640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8dc102160 0x7fb8dc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.783+0000 7fb8e4668640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8dc1017a0 0x7fb8dc197f80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.783+0000 7fb8e4668640 1 --2- 172.21.15.17:0/1691461716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8dc1005b0 0x7fb8dc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:43.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.783+0000 7fb8e4668640 1 -- 172.21.15.17:0/1691461716 >> 172.21.15.17:0/1691461716 conn(0x7fb8dc0fbb40 msgr2=0x7fb8dc104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:43.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.783+0000 7fb8e4668640 1 -- 172.21.15.17:0/1691461716 shutdown_connections 2024-03-24T02:58:43.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:43.783+0000 7fb8e4668640 1 -- 172.21.15.17:0/1691461716 wait complete. 2024-03-24T02:58:43.793 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-24T02:58:43.793 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-24T02:58:43.793 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-24T02:58:44.034 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:58:44.254 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:43 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/579264204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:44.254 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:43 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1413572873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:44.254 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:43 smithi017 ceph-mon[31926]: pgmap v296: 129 pgs: 2 creating+peering, 112 active+clean, 15 unknown; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-03-24T02:58:44.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:43 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/579264204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:44.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:43 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1413572873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:44.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:43 smithi188 ceph-mon[36206]: pgmap v296: 129 pgs: 2 creating+peering, 112 active+clean, 15 unknown; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-03-24T02:58:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:43 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/579264204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:58:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:43 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1413572873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:43 smithi019 ceph-mon[35944]: pgmap v296: 129 pgs: 2 creating+peering, 112 active+clean, 15 unknown; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-03-24T02:58:45.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:44 smithi188 ceph-mon[36206]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T02:58:45.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:44 smithi188 ceph-mon[36206]: Cluster is now healthy 2024-03-24T02:58:45.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:44 smithi019 ceph-mon[35944]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T02:58:45.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:44 smithi019 ceph-mon[35944]: Cluster is now healthy 2024-03-24T02:58:45.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:44 smithi017 ceph-mon[31926]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T02:58:45.357 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:44 smithi017 ceph-mon[31926]: Cluster is now healthy 2024-03-24T02:58:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.609+0000 7f6649d13700 1 -- 172.21.15.17:0/3762562043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66441063a0 msgr2=0x7f664410df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.609+0000 7f6649d13700 1 --2- 172.21.15.17:0/3762562043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66441063a0 0x7f664410df70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f663c00b0a0 tx=0x7f663c00b3b0 comp rx=0 tx=0).stop 2024-03-24T02:58:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.609+0000 7f6649d13700 1 -- 172.21.15.17:0/3762562043 shutdown_connections 2024-03-24T02:58:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.609+0000 7f6649d13700 1 --2- 172.21.15.17:0/3762562043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66441063a0 0x7f664410df70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.609+0000 7f6649d13700 1 --2- 172.21.15.17:0/3762562043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66441012e0 0x7f66441016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.609+0000 7f6649d13700 1 --2- 172.21.15.17:0/3762562043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6644101c90 0x7f6644105cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.609+0000 7f6649d13700 1 -- 172.21.15.17:0/3762562043 >> 172.21.15.17:0/3762562043 conn(0x7f6644078000 msgr2=0x7f6644078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.609+0000 7f6649d13700 1 -- 172.21.15.17:0/3762562043 shutdown_connections 2024-03-24T02:58:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.609+0000 7f6649d13700 1 -- 172.21.15.17:0/3762562043 wait complete. 2024-03-24T02:58:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f6649d13700 1 Processor -- start 2024-03-24T02:58:45.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f6649d13700 1 -- start start 2024-03-24T02:58:45.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f6649d13700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66441012e0 0x7f6644102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:45.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f6649d13700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644101c90 0x7f66441032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:45.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f6649d13700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66441063a0 0x7f66441037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:45.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f6649d13700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6644111380 con 0x7f6644101c90 2024-03-24T02:58:45.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f6649d13700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6644111200 con 0x7f66441063a0 2024-03-24T02:58:45.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f6649d13700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6644111500 con 0x7f66441012e0 2024-03-24T02:58:45.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f6642ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644101c90 0x7f66441032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:45.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f66437fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66441012e0 0x7f6644102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f6642ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644101c90 0x7f66441032a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38982/0 (socket says 172.21.15.17:38982) 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f6642ffd700 1 -- 172.21.15.17:0/3677680856 learned_addr learned my addr 172.21.15.17:0/3677680856 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.610+0000 7f6643fff700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66441063a0 0x7f66441037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.611+0000 7f6642ffd700 1 -- 172.21.15.17:0/3677680856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66441012e0 msgr2=0x7f6644102d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.611+0000 7f6642ffd700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66441012e0 0x7f6644102d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.611+0000 7f6642ffd700 1 -- 172.21.15.17:0/3677680856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66441063a0 msgr2=0x7f66441037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.611+0000 7f6642ffd700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66441063a0 0x7f66441037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.611+0000 7f6642ffd700 1 -- 172.21.15.17:0/3677680856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66441a6d80 con 0x7f6644101c90 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.611+0000 7f6643fff700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66441063a0 0x7f66441037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.611+0000 7f6642ffd700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644101c90 0x7f66441032a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f662c00d980 tx=0x7f662c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.611+0000 7f6640ff9700 1 -- 172.21.15.17:0/3677680856 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c017070 con 0x7f6644101c90 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.611+0000 7f6640ff9700 1 -- 172.21.15.17:0/3677680856 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f662c009e10 con 0x7f6644101c90 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.611+0000 7f6649d13700 1 -- 172.21.15.17:0/3677680856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66441a7070 con 0x7f6644101c90 2024-03-24T02:58:45.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.611+0000 7f6640ff9700 1 -- 172.21.15.17:0/3677680856 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c00f620 con 0x7f6644101c90 2024-03-24T02:58:45.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.611+0000 7f6649d13700 1 -- 172.21.15.17:0/3677680856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66441a75b0 con 0x7f6644101c90 2024-03-24T02:58:45.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.612+0000 7f6649d13700 1 -- 172.21.15.17:0/3677680856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66441027a0 con 0x7f6644101c90 2024-03-24T02:58:45.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.613+0000 7f6640ff9700 1 -- 172.21.15.17:0/3677680856 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f662c010460 con 0x7f6644101c90 2024-03-24T02:58:45.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.614+0000 7f6640ff9700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f66300a2890 0x7f66300a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:45.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.614+0000 7f66437fe700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f66300a2890 0x7f66300a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:45.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.614+0000 7f6640ff9700 1 -- 172.21.15.17:0/3677680856 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f662c0ce790 con 0x7f6644101c90 2024-03-24T02:58:45.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.614+0000 7f66437fe700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f66300a2890 0x7f66300a4d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6634005950 tx=0x7f663400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:45.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.618+0000 7f6640ff9700 1 -- 172.21.15.17:0/3677680856 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f662c09b5f0 con 0x7f6644101c90 2024-03-24T02:58:45.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.873+0000 7f6649d13700 1 -- 172.21.15.17:0/3677680856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f664402da30 con 0x7f6644101c90 2024-03-24T02:58:45.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.873+0000 7f6640ff9700 1 -- 172.21.15.17:0/3677680856 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f662c09aef0 con 0x7f6644101c90 2024-03-24T02:58:45.873 INFO:teuthology.orchestra.run.smithi017.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-24T02:58:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.875+0000 7f6649d13700 1 -- 172.21.15.17:0/3677680856 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f66300a2890 msgr2=0x7f66300a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.875+0000 7f6649d13700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f66300a2890 0x7f66300a4d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6634005950 tx=0x7f663400b410 comp rx=0 tx=0).stop 2024-03-24T02:58:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.875+0000 7f6649d13700 1 -- 172.21.15.17:0/3677680856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644101c90 msgr2=0x7f66441032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.875+0000 7f6649d13700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644101c90 0x7f66441032a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f662c00d980 tx=0x7f662c00dc90 comp rx=0 tx=0).stop 2024-03-24T02:58:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.875+0000 7f6649d13700 1 -- 172.21.15.17:0/3677680856 shutdown_connections 2024-03-24T02:58:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.875+0000 7f6649d13700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f66300a2890 0x7f66300a4d50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.875+0000 7f6649d13700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66441063a0 0x7f66441037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.875+0000 7f6649d13700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66441012e0 0x7f6644102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.875+0000 7f6649d13700 1 --2- 172.21.15.17:0/3677680856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644101c90 0x7f66441032a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.875+0000 7f6649d13700 1 -- 172.21.15.17:0/3677680856 >> 172.21.15.17:0/3677680856 conn(0x7f6644078000 msgr2=0x7f664406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.875+0000 7f6649d13700 1 -- 172.21.15.17:0/3677680856 shutdown_connections 2024-03-24T02:58:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:45.875+0000 7f6649d13700 1 -- 172.21.15.17:0/3677680856 wait complete. 2024-03-24T02:58:46.194 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:45 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3677680856' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T02:58:46.194 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:45 smithi017 ceph-mon[31926]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 4 op/s 2024-03-24T02:58:46.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:45 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3677680856' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T02:58:46.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:45 smithi188 ceph-mon[36206]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 4 op/s 2024-03-24T02:58:46.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:45 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3677680856' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T02:58:46.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:45 smithi019 ceph-mon[35944]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 4 op/s 2024-03-24T02:58:46.530 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-24T02:58:46.530 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-24T02:58:46.530 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi017.front.sepia.ceph.com 2024-03-24T02:58:46.530 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-24T02:58:46.530 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-24T02:58:46.530 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-24T02:58:46.530 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-24T02:58:46.530 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-24T02:58:46.530 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-24T02:58:46.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:46.531 DEBUG:teuthology.orchestra.run.smithi017:> ip addr 2024-03-24T02:58:46.546 INFO:teuthology.orchestra.run.smithi017.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-24T02:58:46.546 INFO:teuthology.orchestra.run.smithi017.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout: inet 127.0.0.1/8 scope host lo 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 ::1/128 scope host 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:00 brd ff:ff:ff:ff:ff:ff 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f0 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:01 brd ff:ff:ff:ff:ff:ff 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f1 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:92 brd ff:ff:ff:ff:ff:ff 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f0 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.21.15.17/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-24T02:58:46.547 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft 42191sec preferred_lft 42191sec 2024-03-24T02:58:46.548 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::ec4:7aff:fe59:d992/64 scope link noprefixroute 2024-03-24T02:58:46.548 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-24T02:58:46.548 INFO:teuthology.orchestra.run.smithi017.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T02:58:46.548 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:93 brd ff:ff:ff:ff:ff:ff 2024-03-24T02:58:46.548 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f1 2024-03-24T02:58:46.548 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-24T02:58:46.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:46.549 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-24T02:58:46.549 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-24T02:58:46.549 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add name ceph-brx type bridge 2024-03-24T02:58:46.549 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr flush dev ceph-brx 2024-03-24T02:58:46.549 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set ceph-brx up 2024-03-24T02:58:46.549 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-24T02:58:46.549 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-24T02:58:46.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:46.725 DEBUG:teuthology.orchestra.run.smithi017:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-24T02:58:46.752 INFO:teuthology.orchestra.run.smithi017.stdout:1 2024-03-24T02:58:46.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:46.754 DEBUG:teuthology.orchestra.run.smithi017:> route 2024-03-24T02:58:46.827 INFO:teuthology.orchestra.run.smithi017.stdout:Kernel IP routing table 2024-03-24T02:58:46.827 INFO:teuthology.orchestra.run.smithi017.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-24T02:58:46.827 INFO:teuthology.orchestra.run.smithi017.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-24T02:58:46.827 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-24T02:58:46.827 INFO:teuthology.orchestra.run.smithi017.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-24T02:58:46.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:46.828 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-24T02:58:46.828 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-24T02:58:46.828 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-24T02:58:46.828 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-24T02:58:46.828 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-24T02:58:46.828 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-24T02:58:47.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:47.120 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-03-24T02:58:47.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:47.135 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list-id 2024-03-24T02:58:47.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:47.189 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-24T02:58:47.190 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-24T02:58:47.190 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-24T02:58:47.190 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-24T02:58:47.190 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-24T02:58:47.271 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-24T02:58:47.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:47.272 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-24T02:58:47.272 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-24T02:58:47.272 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-24T02:58:47.272 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-24T02:58:47.272 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-24T02:58:47.272 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-24T02:58:47.272 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-24T02:58:47.272 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-24T02:58:47.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:47.408 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-24T02:58:47.408 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-24T02:58:47.408 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set brx.0 up 2024-03-24T02:58:47.408 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set dev brx.0 master ceph-brx 2024-03-24T02:58:47.408 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-24T02:58:47.449 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-24T02:58:47.449 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-24T02:58:47.449 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-24T02:58:47.508 INFO:teuthology.orchestra.run.smithi017.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-24T02:58:47.508 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-24T02:58:47.508 DEBUG:teuthology.orchestra.run.smithi017:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-24T02:58:47.562 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-03-24T02:58:47.629 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-03-24T02:58:47.685 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-03-24T02:58:47.685 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-03-24T02:58:47.685 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-03-24T02:58:47.685 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-03-24T02:58:47.685 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-03-24T02:58:47.685 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/selinux 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-03-24T02:58:47.686 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysctl.service 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/nfs/rpc_pipefs 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/a40a8d1b7be11f2da17ca92256050cc3a0426af4f10f176027d77a68a459669b/merged 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/c052a35040f5199efdbae5fa88cfd08f117c902754d6937e31598c1b96bac7c1/merged 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/0 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/262895a4d5f050fc8d6203290502de68a4f10c003cc57e7c41ce55ec03c80fc4/merged 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/f26a6534cb91516afd02fed36371c00ec2022cba8c251692124cc89e6855d4bd/merged 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/a44159c9fabf42f3c48564559dbba643193bd3a7e3fd579f8b0cd0f1b24fbd79/merged 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/ad7e7b76b3bd3bf267d1c3149ee4fa1e2d6cf4db433c20a16a8df4f439b18bf5/merged 2024-03-24T02:58:47.687 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/9eda0d731b86643628d1230ec9c39a660191fe03615b608a00bf9ccbb5c2c96d/merged 2024-03-24T02:58:47.688 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/0e48ef8f18439dcc0fdd4f022cb8ec9326c540fa41d8442cc5d4b69533711288/merged 2024-03-24T02:58:47.688 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/ab08c2d14508f56dbf7180d4dc6a72f55bfa8d5304c08d2ae47132b3f8390881/merged 2024-03-24T02:58:47.688 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/ffde12734b0c169934c89442c0f94e66b61da8a342495853608aee0f6a229b0e/merged 2024-03-24T02:58:47.688 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-03-24T02:58:47.688 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-24T02:58:47.688 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-24T02:58:47.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:47.688 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-03-24T02:58:47.739 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-24T02:58:47.739 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-24T02:58:47.782 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-03-24T02:58:47.808 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-03-24T02:58:47.863 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-03-24T02:58:47.863 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/selinux 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-03-24T02:58:47.864 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysctl.service 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/nfs/rpc_pipefs 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/a40a8d1b7be11f2da17ca92256050cc3a0426af4f10f176027d77a68a459669b/merged 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/c052a35040f5199efdbae5fa88cfd08f117c902754d6937e31598c1b96bac7c1/merged 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/0 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-03-24T02:58:47.865 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/262895a4d5f050fc8d6203290502de68a4f10c003cc57e7c41ce55ec03c80fc4/merged 2024-03-24T02:58:47.866 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/f26a6534cb91516afd02fed36371c00ec2022cba8c251692124cc89e6855d4bd/merged 2024-03-24T02:58:47.866 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/a44159c9fabf42f3c48564559dbba643193bd3a7e3fd579f8b0cd0f1b24fbd79/merged 2024-03-24T02:58:47.866 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/ad7e7b76b3bd3bf267d1c3149ee4fa1e2d6cf4db433c20a16a8df4f439b18bf5/merged 2024-03-24T02:58:47.866 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/9eda0d731b86643628d1230ec9c39a660191fe03615b608a00bf9ccbb5c2c96d/merged 2024-03-24T02:58:47.866 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/0e48ef8f18439dcc0fdd4f022cb8ec9326c540fa41d8442cc5d4b69533711288/merged 2024-03-24T02:58:47.866 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/ab08c2d14508f56dbf7180d4dc6a72f55bfa8d5304c08d2ae47132b3f8390881/merged 2024-03-24T02:58:47.866 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/ffde12734b0c169934c89442c0f94e66b61da8a342495853608aee0f6a229b0e/merged 2024-03-24T02:58:47.866 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-03-24T02:58:47.866 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-24T02:58:47.866 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-24T02:58:47.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:47.867 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-03-24T02:58:47.872 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:2024-03-24T02:58:47.872+0000 7f00d3d71480 -1 init, newargv = 0x55adf669fd90 newargc=15 2024-03-24T02:58:47.872 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[76062]: starting ceph client 2024-03-24T02:58:47.877 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[76062]: starting fuse 2024-03-24T02:58:47.887 INFO:teuthology.orchestra.run.smithi017.stdout:119 2024-03-24T02:58:47.887 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-03-24T02:58:47.887 DEBUG:teuthology.orchestra.run.smithi017:> sudo stdin-killer -- python3 -c ' 2024-03-24T02:58:47.887 DEBUG:teuthology.orchestra.run.smithi017:> import glob 2024-03-24T02:58:47.887 DEBUG:teuthology.orchestra.run.smithi017:> import re 2024-03-24T02:58:47.887 DEBUG:teuthology.orchestra.run.smithi017:> import os 2024-03-24T02:58:47.887 DEBUG:teuthology.orchestra.run.smithi017:> import subprocess 2024-03-24T02:58:47.887 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-24T02:58:47.887 DEBUG:teuthology.orchestra.run.smithi017:> def _find_admin_socket(client_name): 2024-03-24T02:58:47.887 DEBUG:teuthology.orchestra.run.smithi017:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> files = glob.glob(asok_path) 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> # Given a non-glob path, it better be there 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> if "*" not in asok_path: 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> assert(len(files) == 1) 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> return files[0] 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> for f in files: 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> contents = proc_f.read() 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> if mountpoint in contents: 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> return f 2024-03-24T02:58:47.888 DEBUG:teuthology.orchestra.run.smithi017:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-24T02:58:47.889 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-24T02:58:47.889 DEBUG:teuthology.orchestra.run.smithi017:> print(_find_admin_socket("client.admin")) 2024-03-24T02:58:47.889 DEBUG:teuthology.orchestra.run.smithi017:> ' 2024-03-24T02:58:48.009 INFO:teuthology.orchestra.run.smithi017.stdout:/var/run/ceph/ceph-client.admin.76062.asok 2024-03-24T02:58:48.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T02:58:48.019 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76062.asok 2024-03-24T02:58:48.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:48.019 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76062.asok status 2024-03-24T02:58:48.158 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T02:58:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": { 2024-03-24T02:58:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_sha1": "3be9a5c9bc793e11e2800a8c0c696e8b46742033", 2024-03-24T02:58:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-24T02:58:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "entity_id": "admin", 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": "smithi017", 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "pid": "76062", 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "root": "/" 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_count": 0, 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_pinned_count": 0, 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 14787, 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "inst": { 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "name": { 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "client", 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 14787 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-03-24T02:58:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 492258974 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 492258974 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "inst_str": "client.14787 192.168.0.1:0/492258974", 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "addr_str": "192.168.0.1:0/492258974", 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "inode_count": 1, 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_epoch": 17, 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch": 89, 2024-03-24T02:58:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch_barrier": 0, 2024-03-24T02:58:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: "blocklisted": false, 2024-03-24T02:58:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs" 2024-03-24T02:58:48.161 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T02:58:48.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:48.167 DEBUG:teuthology.orchestra.run.smithi017:> stat --file-system '--printf=%T 2024-03-24T02:58:48.167 DEBUG:teuthology.orchestra.run.smithi017:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-24T02:58:48.181 INFO:teuthology.orchestra.run.smithi017.stdout:fuseblk 2024-03-24T02:58:48.182 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-24T02:58:48.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:48.182 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-24T02:58:48.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:48 smithi017 ceph-mon[31926]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 256 B/s wr, 3 op/s 2024-03-24T02:58:48.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:48.233 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-24T02:58:48.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:48 smithi188 ceph-mon[36206]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 256 B/s wr, 3 op/s 2024-03-24T02:58:48.302 INFO:teuthology.orchestra.run.smithi017.stdout:# file: . 2024-03-24T02:58:48.302 INFO:teuthology.orchestra.run.smithi017.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-24T02:58:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:48.302 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:58:48.302 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T02:58:48.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:48 smithi019 ceph-mon[35944]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 256 B/s wr, 3 op/s 2024-03-24T02:58:48.358 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T02:58:48.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.501+0000 7f42c080b640 1 -- 172.21.15.17:0/1166448784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42bc1005c0 msgr2=0x7f42bc1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:48.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.501+0000 7f42c080b640 1 --2- 172.21.15.17:0/1166448784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42bc1005c0 0x7f42bc1009e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009900 tx=0x7f42ac02bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:48.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.501+0000 7f42c080b640 1 -- 172.21.15.17:0/1166448784 shutdown_connections 2024-03-24T02:58:48.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.501+0000 7f42c080b640 1 --2- 172.21.15.17:0/1166448784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc102170 0x7f42bc108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.501+0000 7f42c080b640 1 --2- 172.21.15.17:0/1166448784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc1017b0 0x7f42bc101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.501+0000 7f42c080b640 1 --2- 172.21.15.17:0/1166448784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42bc1005c0 0x7f42bc1009e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.501+0000 7f42c080b640 1 -- 172.21.15.17:0/1166448784 >> 172.21.15.17:0/1166448784 conn(0x7f42bc0fbb30 msgr2=0x7f42bc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:48.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.501+0000 7f42c080b640 1 -- 172.21.15.17:0/1166448784 shutdown_connections 2024-03-24T02:58:48.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.501+0000 7f42c080b640 1 -- 172.21.15.17:0/1166448784 wait complete. 2024-03-24T02:58:48.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42c080b640 1 Processor -- start 2024-03-24T02:58:48.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42c080b640 1 -- start start 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42c080b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc1005c0 0x7f42bc069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42c080b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42bc1017b0 0x7f42bc19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42c080b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc102170 0x7f42bc19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42c080b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc10c230 con 0x7f42bc1017b0 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42c080b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc10c0b0 con 0x7f42bc1005c0 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42c080b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc10c3b0 con 0x7f42bc102170 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42b3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42bc1017b0 0x7f42bc19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42b3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42bc1017b0 0x7f42bc19c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39004/0 (socket says 172.21.15.17:39004) 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42b3fff640 1 -- 172.21.15.17:0/4094804049 learned_addr learned my addr 172.21.15.17:0/4094804049 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42bad9d640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc102170 0x7f42bc19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.502+0000 7f42ba59c640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc1005c0 0x7f42bc069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.503+0000 7f42b3fff640 1 -- 172.21.15.17:0/4094804049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc102170 msgr2=0x7f42bc19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.503+0000 7f42b3fff640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc102170 0x7f42bc19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.503+0000 7f42b3fff640 1 -- 172.21.15.17:0/4094804049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc1005c0 msgr2=0x7f42bc069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:48.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.503+0000 7f42b3fff640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc1005c0 0x7f42bc069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.503+0000 7f42b3fff640 1 -- 172.21.15.17:0/4094804049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc19f660 con 0x7f42bc1017b0 2024-03-24T02:58:48.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.503+0000 7f42b3fff640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42bc1017b0 0x7f42bc19c760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f42a400e9f0 tx=0x7f42a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:48.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.503+0000 7f42b37fe640 1 -- 172.21.15.17:0/4094804049 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a400cd80 con 0x7f42bc1017b0 2024-03-24T02:58:48.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.503+0000 7f42b37fe640 1 -- 172.21.15.17:0/4094804049 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42a4005500 con 0x7f42bc1017b0 2024-03-24T02:58:48.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.503+0000 7f42c080b640 1 -- 172.21.15.17:0/4094804049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc19f950 con 0x7f42bc1017b0 2024-03-24T02:58:48.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.503+0000 7f42b37fe640 1 -- 172.21.15.17:0/4094804049 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4010690 con 0x7f42bc1017b0 2024-03-24T02:58:48.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.503+0000 7f42c080b640 1 -- 172.21.15.17:0/4094804049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc19fea0 con 0x7f42bc1017b0 2024-03-24T02:58:48.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.504+0000 7f42c080b640 1 -- 172.21.15.17:0/4094804049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42bc107980 con 0x7f42bc1017b0 2024-03-24T02:58:48.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.505+0000 7f42b37fe640 1 -- 172.21.15.17:0/4094804049 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f42a40046c0 con 0x7f42bc1017b0 2024-03-24T02:58:48.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.506+0000 7f42b37fe640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f42a80a2880 0x7f42a80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:48.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.506+0000 7f42ba59c640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f42a80a2880 0x7f42a80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:48.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.506+0000 7f42b37fe640 1 -- 172.21.15.17:0/4094804049 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f42a40cf210 con 0x7f42bc1017b0 2024-03-24T02:58:48.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.506+0000 7f42ba59c640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f42a80a2880 0x7f42a80a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f42ac000c00 tx=0x7f42ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:48.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.509+0000 7f42b37fe640 1 -- 172.21.15.17:0/4094804049 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f42a4053740 con 0x7f42bc1017b0 2024-03-24T02:58:48.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.697+0000 7f42c080b640 1 -- 172.21.15.17:0/4094804049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42bc02d070 con 0x7f42bc1017b0 2024-03-24T02:58:48.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.698+0000 7f42b37fe640 1 -- 172.21.15.17:0/4094804049 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24100 (secure 0 0 0) 0x7f42bc02d070 con 0x7f42bc1017b0 2024-03-24T02:58:48.698 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:48.698 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:58:41.346962+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:58:48.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.700+0000 7f42c080b640 1 -- 172.21.15.17:0/4094804049 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f42a80a2880 msgr2=0x7f42a80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:48.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.700+0000 7f42c080b640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f42a80a2880 0x7f42a80a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f42ac000c00 tx=0x7f42ac005fd0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.700+0000 7f42c080b640 1 -- 172.21.15.17:0/4094804049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42bc1017b0 msgr2=0x7f42bc19c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:48.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.700+0000 7f42c080b640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42bc1017b0 0x7f42bc19c760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f42a400e9f0 tx=0x7f42a400edf0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.700+0000 7f42c080b640 1 -- 172.21.15.17:0/4094804049 shutdown_connections 2024-03-24T02:58:48.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.700+0000 7f42c080b640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f42a80a2880 0x7f42a80a4d40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.700+0000 7f42c080b640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc102170 0x7f42bc19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.700+0000 7f42c080b640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42bc1017b0 0x7f42bc19c760 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.701+0000 7f42c080b640 1 --2- 172.21.15.17:0/4094804049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc1005c0 0x7f42bc069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.701+0000 7f42c080b640 1 -- 172.21.15.17:0/4094804049 >> 172.21.15.17:0/4094804049 conn(0x7f42bc0fbb30 msgr2=0x7f42bc104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:48.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.701+0000 7f42c080b640 1 -- 172.21.15.17:0/4094804049 shutdown_connections 2024-03-24T02:58:48.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.701+0000 7f42c080b640 1 -- 172.21.15.17:0/4094804049 wait complete. 2024-03-24T02:58:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T02:53:54.207139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-24T02:58:34.785192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-24T02:58:35.679772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-24T02:58:39.221772+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-24T02:58:48.712 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T02:58:48.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.818+0000 7f5a91c81640 1 -- 172.21.15.17:0/3469508442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a8c100f30 msgr2=0x7f5a8c1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:48.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.818+0000 7f5a91c81640 1 --2- 172.21.15.17:0/3469508442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a8c100f30 0x7f5a8c1013b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009860 tx=0x7f5a8002bb70 comp rx=0 tx=0).stop 2024-03-24T02:58:48.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.818+0000 7f5a91c81640 1 -- 172.21.15.17:0/3469508442 shutdown_connections 2024-03-24T02:58:48.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.818+0000 7f5a91c81640 1 --2- 172.21.15.17:0/3469508442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a8c1018f0 0x7f5a8c108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.818+0000 7f5a91c81640 1 --2- 172.21.15.17:0/3469508442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a8c100f30 0x7f5a8c1013b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.818+0000 7f5a91c81640 1 --2- 172.21.15.17:0/3469508442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c0ffd40 0x7f5a8c100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.818+0000 7f5a91c81640 1 -- 172.21.15.17:0/3469508442 >> 172.21.15.17:0/3469508442 conn(0x7f5a8c0fb290 msgr2=0x7f5a8c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:48.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.818+0000 7f5a91c81640 1 -- 172.21.15.17:0/3469508442 shutdown_connections 2024-03-24T02:58:48.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.818+0000 7f5a91c81640 1 -- 172.21.15.17:0/3469508442 wait complete. 2024-03-24T02:58:48.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a91c81640 1 Processor -- start 2024-03-24T02:58:48.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a91c81640 1 -- start start 2024-03-24T02:58:48.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a91c81640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a8c0ffd40 0x7f5a8c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:48.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a91c81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c100f30 0x7f5a8c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:48.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a91c81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a8c1018f0 0x7f5a8c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:48.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a91c81640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c10d390 con 0x7f5a8c100f30 2024-03-24T02:58:48.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a91c81640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c10d210 con 0x7f5a8c1018f0 2024-03-24T02:58:48.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a91c81640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c10d510 con 0x7f5a8c0ffd40 2024-03-24T02:58:48.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a8affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c100f30 0x7f5a8c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:48.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a8b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a8c0ffd40 0x7f5a8c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:48.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a8affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c100f30 0x7f5a8c197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39018/0 (socket says 172.21.15.17:39018) 2024-03-24T02:58:48.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a8b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a8c0ffd40 0x7f5a8c1979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:54514/0 (socket says 172.21.15.17:54514) 2024-03-24T02:58:48.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.819+0000 7f5a8affd640 1 -- 172.21.15.17:0/4051357823 learned_addr learned my addr 172.21.15.17:0/4051357823 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:48.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a8bfff640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a8c1018f0 0x7f5a8c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:48.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a8b7fe640 1 -- 172.21.15.17:0/4051357823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a8c1018f0 msgr2=0x7f5a8c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:48.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a8b7fe640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a8c1018f0 0x7f5a8c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a8b7fe640 1 -- 172.21.15.17:0/4051357823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c100f30 msgr2=0x7f5a8c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:48.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a8b7fe640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c100f30 0x7f5a8c197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:48.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a8b7fe640 1 -- 172.21.15.17:0/4051357823 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c19f5f0 con 0x7f5a8c0ffd40 2024-03-24T02:58:48.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a8bfff640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a8c1018f0 0x7f5a8c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:58:48.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a8affd640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c100f30 0x7f5a8c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:58:48.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a8b7fe640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a8c0ffd40 0x7f5a8c1979d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c100d90 tx=0x7f5a7c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:48.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a88ff9640 1 -- 172.21.15.17:0/4051357823 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c004440 con 0x7f5a8c0ffd40 2024-03-24T02:58:48.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a91c81640 1 -- 172.21.15.17:0/4051357823 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8c19f8e0 con 0x7f5a8c0ffd40 2024-03-24T02:58:48.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a88ff9640 1 -- 172.21.15.17:0/4051357823 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a7c00cda0 con 0x7f5a8c0ffd40 2024-03-24T02:58:48.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a88ff9640 1 -- 172.21.15.17:0/4051357823 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c0196e0 con 0x7f5a8c0ffd40 2024-03-24T02:58:48.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.820+0000 7f5a91c81640 1 -- 172.21.15.17:0/4051357823 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8c19fe30 con 0x7f5a8c0ffd40 2024-03-24T02:58:48.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.821+0000 7f5a91c81640 1 -- 172.21.15.17:0/4051357823 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8c107080 con 0x7f5a8c0ffd40 2024-03-24T02:58:48.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.822+0000 7f5a88ff9640 1 -- 172.21.15.17:0/4051357823 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5a7c003780 con 0x7f5a8c0ffd40 2024-03-24T02:58:48.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.823+0000 7f5a88ff9640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a740a2800 0x7f5a740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:48.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.823+0000 7f5a8affd640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a740a2800 0x7f5a740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:48.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.823+0000 7f5a88ff9640 1 -- 172.21.15.17:0/4051357823 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f5a7c0cf230 con 0x7f5a8c0ffd40 2024-03-24T02:58:48.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.823+0000 7f5a8affd640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a740a2800 0x7f5a740a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c198ee0 tx=0x7f5a8002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:48.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:48.831+0000 7f5a88ff9640 1 -- 172.21.15.17:0/4051357823 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a7c009510 con 0x7f5a8c0ffd40 2024-03-24T02:58:49.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.018+0000 7f5a91c81640 1 -- 172.21.15.17:0/4051357823 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a8c0622a0 con 0x7f5a8c0ffd40 2024-03-24T02:58:49.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.018+0000 7f5a88ff9640 1 -- 172.21.15.17:0/4051357823 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a7c003e20 con 0x7f5a8c0ffd40 2024-03-24T02:58:49.018 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-24T02:58:49.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.021+0000 7f5a91c81640 1 -- 172.21.15.17:0/4051357823 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a740a2800 msgr2=0x7f5a740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.021+0000 7f5a91c81640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a740a2800 0x7f5a740a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c198ee0 tx=0x7f5a8002e040 comp rx=0 tx=0).stop 2024-03-24T02:58:49.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.021+0000 7f5a91c81640 1 -- 172.21.15.17:0/4051357823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a8c0ffd40 msgr2=0x7f5a8c1979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.021+0000 7f5a91c81640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a8c0ffd40 0x7f5a8c1979d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c100d90 tx=0x7f5a7c00ee50 comp rx=0 tx=0).stop 2024-03-24T02:58:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.021+0000 7f5a91c81640 1 -- 172.21.15.17:0/4051357823 shutdown_connections 2024-03-24T02:58:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.021+0000 7f5a91c81640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a740a2800 0x7f5a740a4cc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.021+0000 7f5a91c81640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a8c1018f0 0x7f5a8c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.021+0000 7f5a91c81640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c100f30 0x7f5a8c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.021+0000 7f5a91c81640 1 --2- 172.21.15.17:0/4051357823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a8c0ffd40 0x7f5a8c1979d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.021+0000 7f5a91c81640 1 -- 172.21.15.17:0/4051357823 >> 172.21.15.17:0/4051357823 conn(0x7f5a8c0fb290 msgr2=0x7f5a8c104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.021+0000 7f5a91c81640 1 -- 172.21.15.17:0/4051357823 shutdown_connections 2024-03-24T02:58:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.021+0000 7f5a91c81640 1 -- 172.21.15.17:0/4051357823 wait complete. 2024-03-24T02:58:49.031 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-24T02:58:49.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.137+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/235843406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f18104a70 msgr2=0x7f8f18106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.137+0000 7f8f1e2c6640 1 --2- 172.21.15.17:0/235843406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f18104a70 0x7f8f18106e60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009900 tx=0x7f8f0c02bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.137+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/235843406 shutdown_connections 2024-03-24T02:58:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.137+0000 7f8f1e2c6640 1 --2- 172.21.15.17:0/235843406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f181073a0 0x7f8f181097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.137+0000 7f8f1e2c6640 1 --2- 172.21.15.17:0/235843406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f18104a70 0x7f8f18106e60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.137+0000 7f8f1e2c6640 1 --2- 172.21.15.17:0/235843406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f18102140 0x7f8f18104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.137+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/235843406 >> 172.21.15.17:0/235843406 conn(0x7f8f180fbb50 msgr2=0x7f8f180fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.137+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/235843406 shutdown_connections 2024-03-24T02:58:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.137+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/235843406 wait complete. 2024-03-24T02:58:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.137+0000 7f8f1e2c6640 1 Processor -- start 2024-03-24T02:58:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.137+0000 7f8f1e2c6640 1 -- start start 2024-03-24T02:58:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1e2c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f18102140 0x7f8f18101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1e2c6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f18104a70 0x7f8f181000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1e2c6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f181073a0 0x7f8f18100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1e2c6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1810d340 con 0x7f8f181073a0 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1e2c6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1810d1c0 con 0x7f8f18102140 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1e2c6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1810d4c0 con 0x7f8f18104a70 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1c863640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f181073a0 0x7f8f18100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1c863640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f181073a0 0x7f8f18100630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39038/0 (socket says 172.21.15.17:39038) 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1c863640 1 -- 172.21.15.17:0/477065235 learned_addr learned my addr 172.21.15.17:0/477065235 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f177fe640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f18104a70 0x7f8f181000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f17fff640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f18102140 0x7f8f18101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1c863640 1 -- 172.21.15.17:0/477065235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f18104a70 msgr2=0x7f8f181000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1c863640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f18104a70 0x7f8f181000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1c863640 1 -- 172.21.15.17:0/477065235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f18102140 msgr2=0x7f8f18101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1c863640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f18102140 0x7f8f18101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1c863640 1 -- 172.21.15.17:0/477065235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f18100ec0 con 0x7f8f181073a0 2024-03-24T02:58:49.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f17fff640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f18102140 0x7f8f18101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:58:49.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.138+0000 7f8f1c863640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f181073a0 0x7f8f18100630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000eb10 tx=0x7f8f1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:49.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.139+0000 7f8f157fa640 1 -- 172.21.15.17:0/477065235 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f10004b20 con 0x7f8f181073a0 2024-03-24T02:58:49.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.139+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/477065235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f181a1b30 con 0x7f8f181073a0 2024-03-24T02:58:49.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.139+0000 7f8f157fa640 1 -- 172.21.15.17:0/477065235 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f10004cc0 con 0x7f8f181073a0 2024-03-24T02:58:49.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.139+0000 7f8f157fa640 1 -- 172.21.15.17:0/477065235 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f10010760 con 0x7f8f181073a0 2024-03-24T02:58:49.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.139+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/477065235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f181a20a0 con 0x7f8f181073a0 2024-03-24T02:58:49.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.140+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/477065235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f180fd640 con 0x7f8f181073a0 2024-03-24T02:58:49.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.141+0000 7f8f157fa640 1 -- 172.21.15.17:0/477065235 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8f10005020 con 0x7f8f181073a0 2024-03-24T02:58:49.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.141+0000 7f8f157fa640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8f000a2800 0x7f8f000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:49.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.141+0000 7f8f17fff640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8f000a2800 0x7f8f000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:49.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.142+0000 7f8f157fa640 1 -- 172.21.15.17:0/477065235 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8f100cf4b0 con 0x7f8f181073a0 2024-03-24T02:58:49.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.142+0000 7f8f17fff640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8f000a2800 0x7f8f000a4cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009b40 tx=0x7f8f08005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:49.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.146+0000 7f8f157fa640 1 -- 172.21.15.17:0/477065235 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8f1009c2b0 con 0x7f8f181073a0 2024-03-24T02:58:49.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:49 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4094804049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:49.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:49 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4094804049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:49.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.330+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/477065235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f180622a0 con 0x7f8f181073a0 2024-03-24T02:58:49.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.331+0000 7f8f157fa640 1 -- 172.21.15.17:0/477065235 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f1009bbb0 con 0x7f8f181073a0 2024-03-24T02:58:49.330 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T02:58:49.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.333+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/477065235 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8f000a2800 msgr2=0x7f8f000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.333+0000 7f8f1e2c6640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8f000a2800 0x7f8f000a4cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009b40 tx=0x7f8f08005e50 comp rx=0 tx=0).stop 2024-03-24T02:58:49.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.333+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/477065235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f181073a0 msgr2=0x7f8f18100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.333+0000 7f8f1e2c6640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f181073a0 0x7f8f18100630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000eb10 tx=0x7f8f1000c2d0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.333+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/477065235 shutdown_connections 2024-03-24T02:58:49.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.333+0000 7f8f1e2c6640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8f000a2800 0x7f8f000a4cc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.333+0000 7f8f1e2c6640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f181073a0 0x7f8f18100630 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.333+0000 7f8f1e2c6640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f18104a70 0x7f8f181000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.333+0000 7f8f1e2c6640 1 --2- 172.21.15.17:0/477065235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f18102140 0x7f8f18101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.333+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/477065235 >> 172.21.15.17:0/477065235 conn(0x7f8f180fbb50 msgr2=0x7f8f18102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:49.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.333+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/477065235 shutdown_connections 2024-03-24T02:58:49.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.333+0000 7f8f1e2c6640 1 -- 172.21.15.17:0/477065235 wait complete. 2024-03-24T02:58:49.341 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:49 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4094804049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:49.343 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-24T02:58:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.448+0000 7fe827dd4640 1 -- 172.21.15.17:0/2457390411 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8201016b0 msgr2=0x7fe820101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.448+0000 7fe827dd4640 1 --2- 172.21.15.17:0/2457390411 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8201016b0 0x7fe820101ad0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe818009900 tx=0x7fe81802bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.448+0000 7fe827dd4640 1 -- 172.21.15.17:0/2457390411 shutdown_connections 2024-03-24T02:58:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.448+0000 7fe827dd4640 1 --2- 172.21.15.17:0/2457390411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe820103260 0x7fe820109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.448+0000 7fe827dd4640 1 --2- 172.21.15.17:0/2457390411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8201028a0 0x7fe820102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.448+0000 7fe827dd4640 1 --2- 172.21.15.17:0/2457390411 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8201016b0 0x7fe820101ad0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe827dd4640 1 -- 172.21.15.17:0/2457390411 >> 172.21.15.17:0/2457390411 conn(0x7fe8200fcc40 msgr2=0x7fe8200ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe827dd4640 1 -- 172.21.15.17:0/2457390411 shutdown_connections 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe827dd4640 1 -- 172.21.15.17:0/2457390411 wait complete. 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe827dd4640 1 Processor -- start 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe827dd4640 1 -- start start 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe827dd4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8201016b0 0x7fe820197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe827dd4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8201028a0 0x7fe820197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe827dd4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820103260 0x7fe82019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe827dd4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82010d300 con 0x7fe8201016b0 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe827dd4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82010d180 con 0x7fe820103260 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe827dd4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82010d480 con 0x7fe8201028a0 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe825b70640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8201016b0 0x7fe820197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe825b70640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8201016b0 0x7fe820197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39066/0 (socket says 172.21.15.17:39066) 2024-03-24T02:58:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.449+0000 7fe825b70640 1 -- 172.21.15.17:0/2138516101 learned_addr learned my addr 172.21.15.17:0/2138516101 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe82536f640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8201028a0 0x7fe820197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe826371640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820103260 0x7fe82019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe825b70640 1 -- 172.21.15.17:0/2138516101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8201028a0 msgr2=0x7fe820197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe825b70640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8201028a0 0x7fe820197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe825b70640 1 -- 172.21.15.17:0/2138516101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820103260 msgr2=0x7fe82019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe825b70640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820103260 0x7fe82019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe825b70640 1 -- 172.21.15.17:0/2138516101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82019f620 con 0x7fe8201016b0 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe825b70640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8201016b0 0x7fe820197a00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe818000c00 tx=0x7fe818005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe81effd640 1 -- 172.21.15.17:0/2138516101 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818041070 con 0x7fe8201016b0 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe81effd640 1 -- 172.21.15.17:0/2138516101 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8180044d0 con 0x7fe8201016b0 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe81effd640 1 -- 172.21.15.17:0/2138516101 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818034440 con 0x7fe8201016b0 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe827dd4640 1 -- 172.21.15.17:0/2138516101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82019f8b0 con 0x7fe8201016b0 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.450+0000 7fe827dd4640 1 -- 172.21.15.17:0/2138516101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82019fd50 con 0x7fe8201016b0 2024-03-24T02:58:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.451+0000 7fe827dd4640 1 -- 172.21.15.17:0/2138516101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe820108a40 con 0x7fe8201016b0 2024-03-24T02:58:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.452+0000 7fe81effd640 1 -- 172.21.15.17:0/2138516101 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe818044070 con 0x7fe8201016b0 2024-03-24T02:58:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.453+0000 7fe81effd640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe8080a2800 0x7fe8080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.453+0000 7fe82536f640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe8080a2800 0x7fe8080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.453+0000 7fe81effd640 1 -- 172.21.15.17:0/2138516101 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe8180f0ac0 con 0x7fe8201016b0 2024-03-24T02:58:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.453+0000 7fe82536f640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe8080a2800 0x7fe8080a4cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe820198f10 tx=0x7fe81400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.456+0000 7fe81effd640 1 -- 172.21.15.17:0/2138516101 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe8180bd8c0 con 0x7fe8201016b0 2024-03-24T02:58:49.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.639+0000 7fe827dd4640 1 -- 172.21.15.17:0/2138516101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe82002d200 con 0x7fe8201016b0 2024-03-24T02:58:49.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.639+0000 7fe81effd640 1 -- 172.21.15.17:0/2138516101 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe8180bd1c0 con 0x7fe8201016b0 2024-03-24T02:58:49.639 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-24T02:58:49.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.641+0000 7fe827dd4640 1 -- 172.21.15.17:0/2138516101 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe8080a2800 msgr2=0x7fe8080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.641+0000 7fe827dd4640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe8080a2800 0x7fe8080a4cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe820198f10 tx=0x7fe81400b410 comp rx=0 tx=0).stop 2024-03-24T02:58:49.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.641+0000 7fe827dd4640 1 -- 172.21.15.17:0/2138516101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8201016b0 msgr2=0x7fe820197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.641+0000 7fe827dd4640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8201016b0 0x7fe820197a00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe818000c00 tx=0x7fe818005d50 comp rx=0 tx=0).stop 2024-03-24T02:58:49.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.641+0000 7fe827dd4640 1 -- 172.21.15.17:0/2138516101 shutdown_connections 2024-03-24T02:58:49.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.641+0000 7fe827dd4640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe8080a2800 0x7fe8080a4cc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.641+0000 7fe827dd4640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820103260 0x7fe82019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.641+0000 7fe827dd4640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8201028a0 0x7fe820197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.641+0000 7fe827dd4640 1 --2- 172.21.15.17:0/2138516101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8201016b0 0x7fe820197a00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.641+0000 7fe827dd4640 1 -- 172.21.15.17:0/2138516101 >> 172.21.15.17:0/2138516101 conn(0x7fe8200fcc40 msgr2=0x7fe820105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:49.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.642+0000 7fe827dd4640 1 -- 172.21.15.17:0/2138516101 shutdown_connections 2024-03-24T02:58:49.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.642+0000 7fe827dd4640 1 -- 172.21.15.17:0/2138516101 wait complete. 2024-03-24T02:58:49.652 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-24T02:58:49.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.758+0000 7f2eae668640 1 -- 172.21.15.17:0/3639086163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea81021a0 msgr2=0x7f2ea8104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.758+0000 7f2eae668640 1 --2- 172.21.15.17:0/3639086163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea81021a0 0x7f2ea8104590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009900 tx=0x7f2e9802bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:49.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.758+0000 7f2eae668640 1 -- 172.21.15.17:0/3639086163 shutdown_connections 2024-03-24T02:58:49.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.758+0000 7f2eae668640 1 --2- 172.21.15.17:0/3639086163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea8107400 0x7f2ea81097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.758+0000 7f2eae668640 1 --2- 172.21.15.17:0/3639086163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea8104ad0 0x7f2ea8106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.758+0000 7f2eae668640 1 --2- 172.21.15.17:0/3639086163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea81021a0 0x7f2ea8104590 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.758+0000 7f2eae668640 1 -- 172.21.15.17:0/3639086163 >> 172.21.15.17:0/3639086163 conn(0x7f2ea80fbb50 msgr2=0x7f2ea80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.758+0000 7f2eae668640 1 -- 172.21.15.17:0/3639086163 shutdown_connections 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.758+0000 7f2eae668640 1 -- 172.21.15.17:0/3639086163 wait complete. 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.758+0000 7f2eae668640 1 Processor -- start 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.758+0000 7f2eae668640 1 -- start start 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2eae668640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea81021a0 0x7f2ea81979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2eae668640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea8104ad0 0x7f2ea8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2eae668640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea8107400 0x7f2ea819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2eae668640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea810d360 con 0x7f2ea8104ad0 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2eae668640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea810d1e0 con 0x7f2ea81021a0 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2eae668640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea810d4e0 con 0x7f2ea8107400 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea8104ad0 0x7f2ea8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:49.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea8104ad0 0x7f2ea8197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39090/0 (socket says 172.21.15.17:39090) 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea77fe640 1 -- 172.21.15.17:0/762404752 learned_addr learned my addr 172.21.15.17:0/762404752 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2eacc05640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea8107400 0x7f2ea819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea7fff640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea81021a0 0x7f2ea81979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea77fe640 1 -- 172.21.15.17:0/762404752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea8107400 msgr2=0x7f2ea819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea77fe640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea8107400 0x7f2ea819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea77fe640 1 -- 172.21.15.17:0/762404752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea81021a0 msgr2=0x7f2ea81979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea77fe640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea81021a0 0x7f2ea81979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea77fe640 1 -- 172.21.15.17:0/762404752 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea819f5d0 con 0x7f2ea8104ad0 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea7fff640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea81021a0 0x7f2ea81979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea77fe640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea8104ad0 0x7f2ea8197ef0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00e9f0 tx=0x7f2e9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea57fa640 1 -- 172.21.15.17:0/762404752 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00ccc0 con 0x7f2ea8104ad0 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2eae668640 1 -- 172.21.15.17:0/762404752 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea819f8c0 con 0x7f2ea8104ad0 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea57fa640 1 -- 172.21.15.17:0/762404752 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e9c005500 con 0x7f2ea8104ad0 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.759+0000 7f2ea57fa640 1 -- 172.21.15.17:0/762404752 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c010690 con 0x7f2ea8104ad0 2024-03-24T02:58:49.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.760+0000 7f2eae668640 1 -- 172.21.15.17:0/762404752 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea819fdc0 con 0x7f2ea8104ad0 2024-03-24T02:58:49.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.760+0000 7f2eae668640 1 -- 172.21.15.17:0/762404752 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea80fd640 con 0x7f2ea8104ad0 2024-03-24T02:58:49.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.762+0000 7f2ea57fa640 1 -- 172.21.15.17:0/762404752 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2e9c0046c0 con 0x7f2ea8104ad0 2024-03-24T02:58:49.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.762+0000 7f2ea57fa640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2e900a2800 0x7f2e900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:49.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.762+0000 7f2ea7fff640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2e900a2800 0x7f2e900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:49.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.762+0000 7f2ea57fa640 1 -- 172.21.15.17:0/762404752 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2e9c0cf0b0 con 0x7f2ea8104ad0 2024-03-24T02:58:49.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.763+0000 7f2ea7fff640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2e900a2800 0x7f2e900a4cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2e980096c0 tx=0x7f2e98000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:49.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.766+0000 7f2ea57fa640 1 -- 172.21.15.17:0/762404752 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2e9c09bf10 con 0x7f2ea8104ad0 2024-03-24T02:58:49.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.949+0000 7f2eae668640 1 -- 172.21.15.17:0/762404752 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2ea80622a0 con 0x7f2ea8104ad0 2024-03-24T02:58:49.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.949+0000 7f2ea57fa640 1 -- 172.21.15.17:0/762404752 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2e9c09b810 con 0x7f2ea8104ad0 2024-03-24T02:58:49.949 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T02:58:49.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.951+0000 7f2eae668640 1 -- 172.21.15.17:0/762404752 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2e900a2800 msgr2=0x7f2e900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.951+0000 7f2eae668640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2e900a2800 0x7f2e900a4cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2e980096c0 tx=0x7f2e98000bc0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.951+0000 7f2eae668640 1 -- 172.21.15.17:0/762404752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea8104ad0 msgr2=0x7f2ea8197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:49.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.951+0000 7f2eae668640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea8104ad0 0x7f2ea8197ef0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00e9f0 tx=0x7f2e9c00eeb0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.951+0000 7f2eae668640 1 -- 172.21.15.17:0/762404752 shutdown_connections 2024-03-24T02:58:49.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.951+0000 7f2eae668640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2e900a2800 0x7f2e900a4cc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.951+0000 7f2eae668640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea8107400 0x7f2ea819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.951+0000 7f2eae668640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea8104ad0 0x7f2ea8197ef0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.951+0000 7f2eae668640 1 --2- 172.21.15.17:0/762404752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea81021a0 0x7f2ea81979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:49.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.951+0000 7f2eae668640 1 -- 172.21.15.17:0/762404752 >> 172.21.15.17:0/762404752 conn(0x7f2ea80fbb50 msgr2=0x7f2ea8102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:49.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.952+0000 7f2eae668640 1 -- 172.21.15.17:0/762404752 shutdown_connections 2024-03-24T02:58:49.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:49.952+0000 7f2eae668640 1 -- 172.21.15.17:0/762404752 wait complete. 2024-03-24T02:58:49.962 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-24T02:58:50.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.069+0000 7f7cd7d16640 1 -- 172.21.15.17:0/2004216217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd0104ab0 msgr2=0x7f7cd0106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.069+0000 7f7cd7d16640 1 --2- 172.21.15.17:0/2004216217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd0104ab0 0x7f7cd0106ea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009860 tx=0x7f7cc402bb70 comp rx=0 tx=0).stop 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.069+0000 7f7cd7d16640 1 -- 172.21.15.17:0/2004216217 shutdown_connections 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.069+0000 7f7cd7d16640 1 --2- 172.21.15.17:0/2004216217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cd01073e0 0x7f7cd01097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.069+0000 7f7cd7d16640 1 --2- 172.21.15.17:0/2004216217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd0104ab0 0x7f7cd0106ea0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.069+0000 7f7cd7d16640 1 --2- 172.21.15.17:0/2004216217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd0102180 0x7f7cd0104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.069+0000 7f7cd7d16640 1 -- 172.21.15.17:0/2004216217 >> 172.21.15.17:0/2004216217 conn(0x7f7cd00fbb30 msgr2=0x7f7cd00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.069+0000 7f7cd7d16640 1 -- 172.21.15.17:0/2004216217 shutdown_connections 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd7d16640 1 -- 172.21.15.17:0/2004216217 wait complete. 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd7d16640 1 Processor -- start 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd7d16640 1 -- start start 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd7d16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd0102180 0x7f7cd0101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd7d16640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd0104ab0 0x7f7cd0100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd7d16640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cd01073e0 0x7f7cd0100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:50.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd7d16640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd010d300 con 0x7f7cd01073e0 2024-03-24T02:58:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd7d16640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd010d180 con 0x7f7cd0102180 2024-03-24T02:58:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd7d16640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd010d480 con 0x7f7cd0104ab0 2024-03-24T02:58:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd62b3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cd01073e0 0x7f7cd0100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd62b3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cd01073e0 0x7f7cd0100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39106/0 (socket says 172.21.15.17:39106) 2024-03-24T02:58:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd62b3640 1 -- 172.21.15.17:0/1897484316 learned_addr learned my addr 172.21.15.17:0/1897484316 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd52b1640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd0104ab0 0x7f7cd0100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.070+0000 7f7cd5ab2640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd0102180 0x7f7cd0101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.071+0000 7f7cd62b3640 1 -- 172.21.15.17:0/1897484316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd0104ab0 msgr2=0x7f7cd0100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.071+0000 7f7cd62b3640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd0104ab0 0x7f7cd0100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.071+0000 7f7cd62b3640 1 -- 172.21.15.17:0/1897484316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd0102180 msgr2=0x7f7cd0101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.071+0000 7f7cd62b3640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd0102180 0x7f7cd0101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.071+0000 7f7cd62b3640 1 -- 172.21.15.17:0/1897484316 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd0100f00 con 0x7f7cd01073e0 2024-03-24T02:58:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.071+0000 7f7cd52b1640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd0104ab0 0x7f7cd0100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:58:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.071+0000 7f7cd62b3640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cd01073e0 0x7f7cd0100670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000e940 tx=0x7f7cc000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.071+0000 7f7cceffd640 1 -- 172.21.15.17:0/1897484316 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc000cd80 con 0x7f7cd01073e0 2024-03-24T02:58:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.071+0000 7f7cd7d16640 1 -- 172.21.15.17:0/1897484316 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd01a1ac0 con 0x7f7cd01073e0 2024-03-24T02:58:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.071+0000 7f7cceffd640 1 -- 172.21.15.17:0/1897484316 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cc00035f0 con 0x7f7cd01073e0 2024-03-24T02:58:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.071+0000 7f7cceffd640 1 -- 172.21.15.17:0/1897484316 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0010690 con 0x7f7cd01073e0 2024-03-24T02:58:50.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.071+0000 7f7cd7d16640 1 -- 172.21.15.17:0/1897484316 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd01a1fe0 con 0x7f7cd01073e0 2024-03-24T02:58:50.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.072+0000 7f7cd7d16640 1 -- 172.21.15.17:0/1897484316 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd0068490 con 0x7f7cd01073e0 2024-03-24T02:58:50.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.073+0000 7f7cceffd640 1 -- 172.21.15.17:0/1897484316 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7cc00037a0 con 0x7f7cd01073e0 2024-03-24T02:58:50.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.074+0000 7f7cceffd640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7cb80a2800 0x7f7cb80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:50.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.074+0000 7f7cd5ab2640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7cb80a2800 0x7f7cb80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:50.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.074+0000 7f7cceffd640 1 -- 172.21.15.17:0/1897484316 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f7cc00cf150 con 0x7f7cd01073e0 2024-03-24T02:58:50.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.074+0000 7f7cd5ab2640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7cb80a2800 0x7f7cb80a4cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8007ec0 tx=0x7f7cc80075b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:50.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.077+0000 7f7cceffd640 1 -- 172.21.15.17:0/1897484316 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7cc009bfe0 con 0x7f7cd01073e0 2024-03-24T02:58:50.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.273+0000 7f7cd7d16640 1 -- 172.21.15.17:0/1897484316 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7cd00622a0 con 0x7f7cd01073e0 2024-03-24T02:58:50.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.273+0000 7f7cceffd640 1 -- 172.21.15.17:0/1897484316 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+586 (secure 0 0 0) 0x7f7cc009b8e0 con 0x7f7cd01073e0 2024-03-24T02:58:50.273 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/1870031079 2024-03-25T02:51:29.864563+0000 2024-03-24T02:58:50.273 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/3007657427 2024-03-25T02:51:29.864563+0000 2024-03-24T02:58:50.273 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/3007657427 2024-03-25T02:51:29.864563+0000 2024-03-24T02:58:50.273 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/2226886960 2024-03-25T02:51:02.292414+0000 2024-03-24T02:58:50.273 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/4291784529 2024-03-25T02:51:02.292414+0000 2024-03-24T02:58:50.273 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/4202143244 2024-03-25T02:51:02.292414+0000 2024-03-24T02:58:50.273 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/4291784529 2024-03-25T02:51:02.292414+0000 2024-03-24T02:58:50.273 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/630595937 2024-03-25T02:51:02.292414+0000 2024-03-24T02:58:50.273 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/532405 2024-03-25T02:51:29.864563+0000 2024-03-24T02:58:50.273 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/467902990 2024-03-25T02:51:29.864563+0000 2024-03-24T02:58:50.274 INFO:teuthology.orchestra.run.smithi017.stderr:listed 10 entries 2024-03-24T02:58:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.275+0000 7f7cd7d16640 1 -- 172.21.15.17:0/1897484316 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7cb80a2800 msgr2=0x7f7cb80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.275+0000 7f7cd7d16640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7cb80a2800 0x7f7cb80a4cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8007ec0 tx=0x7f7cc80075b0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.275+0000 7f7cd7d16640 1 -- 172.21.15.17:0/1897484316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cd01073e0 msgr2=0x7f7cd0100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.275+0000 7f7cd7d16640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cd01073e0 0x7f7cd0100670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000e940 tx=0x7f7cc000ee00 comp rx=0 tx=0).stop 2024-03-24T02:58:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.275+0000 7f7cd7d16640 1 -- 172.21.15.17:0/1897484316 shutdown_connections 2024-03-24T02:58:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.275+0000 7f7cd7d16640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7cb80a2800 0x7f7cb80a4cc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.275+0000 7f7cd7d16640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cd01073e0 0x7f7cd0100670 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.275+0000 7f7cd7d16640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd0104ab0 0x7f7cd0100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.275+0000 7f7cd7d16640 1 --2- 172.21.15.17:0/1897484316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd0102180 0x7f7cd0101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.275+0000 7f7cd7d16640 1 -- 172.21.15.17:0/1897484316 >> 172.21.15.17:0/1897484316 conn(0x7f7cd00fbb30 msgr2=0x7f7cd0102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.275+0000 7f7cd7d16640 1 -- 172.21.15.17:0/1897484316 shutdown_connections 2024-03-24T02:58:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.275+0000 7f7cd7d16640 1 -- 172.21.15.17:0/1897484316 wait complete. 2024-03-24T02:58:50.284 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:50 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4051357823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:58:50.284 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:50 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/477065235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T02:58:50.284 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:50 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2138516101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T02:58:50.284 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:50 smithi017 ceph-mon[31926]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 114 B/s wr, 0 op/s 2024-03-24T02:58:50.285 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:50 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/762404752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T02:58:50.285 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:58:50.285 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T02:58:50.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:50 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4051357823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:58:50.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:50 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/477065235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T02:58:50.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:50 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2138516101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T02:58:50.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:50 smithi188 ceph-mon[36206]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 114 B/s wr, 0 op/s 2024-03-24T02:58:50.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:50 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/762404752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T02:58:50.300 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T02:58:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:50 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4051357823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:58:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:50 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/477065235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T02:58:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:50 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2138516101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T02:58:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:50 smithi019 ceph-mon[35944]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 114 B/s wr, 0 op/s 2024-03-24T02:58:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:50 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/762404752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T02:58:50.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.444+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3137680434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c681028b0 msgr2=0x7f4c68102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.444+0000 7f4c6e84b640 1 --2- 172.21.15.17:0/3137680434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c681028b0 0x7f4c68102d30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009900 tx=0x7f4c5c02bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:50.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.444+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3137680434 shutdown_connections 2024-03-24T02:58:50.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.444+0000 7f4c6e84b640 1 --2- 172.21.15.17:0/3137680434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c68103270 0x7f4c68109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.444+0000 7f4c6e84b640 1 --2- 172.21.15.17:0/3137680434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c681028b0 0x7f4c68102d30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.444+0000 7f4c6e84b640 1 --2- 172.21.15.17:0/3137680434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681016c0 0x7f4c68101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.444+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3137680434 >> 172.21.15.17:0/3137680434 conn(0x7f4c680fcc50 msgr2=0x7f4c680ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:50.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.444+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3137680434 shutdown_connections 2024-03-24T02:58:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.444+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3137680434 wait complete. 2024-03-24T02:58:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.444+0000 7f4c6e84b640 1 Processor -- start 2024-03-24T02:58:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.444+0000 7f4c6e84b640 1 -- start start 2024-03-24T02:58:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c6e84b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c681016c0 0x7f4c68195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c6e84b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681028b0 0x7f4c68195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c6e84b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c68103270 0x7f4c6819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c6e84b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6810b160 con 0x7f4c681016c0 2024-03-24T02:58:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c6e84b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6810afe0 con 0x7f4c681028b0 2024-03-24T02:58:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c6e84b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6810b2e0 con 0x7f4c68103270 2024-03-24T02:58:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c681016c0 0x7f4c68195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c681016c0 0x7f4c68195770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39130/0 (socket says 172.21.15.17:39130) 2024-03-24T02:58:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c67fff640 1 -- 172.21.15.17:0/3203873701 learned_addr learned my addr 172.21.15.17:0/3203873701 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c6cde8640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c68103270 0x7f4c6819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c677fe640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681028b0 0x7f4c68195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c67fff640 1 -- 172.21.15.17:0/3203873701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c68103270 msgr2=0x7f4c6819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c67fff640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c68103270 0x7f4c6819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c67fff640 1 -- 172.21.15.17:0/3203873701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681028b0 msgr2=0x7f4c68195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c67fff640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681028b0 0x7f4c68195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c67fff640 1 -- 172.21.15.17:0/3203873701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6819d390 con 0x7f4c681016c0 2024-03-24T02:58:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c677fe640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681028b0 0x7f4c68195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:58:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c67fff640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c681016c0 0x7f4c68195770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4c58011a60 tx=0x7f4c58011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c657fa640 1 -- 172.21.15.17:0/3203873701 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58004220 con 0x7f4c681016c0 2024-03-24T02:58:50.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c657fa640 1 -- 172.21.15.17:0/3203873701 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c580043c0 con 0x7f4c681016c0 2024-03-24T02:58:50.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3203873701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6819d680 con 0x7f4c681016c0 2024-03-24T02:58:50.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.445+0000 7f4c657fa640 1 -- 172.21.15.17:0/3203873701 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58019660 con 0x7f4c681016c0 2024-03-24T02:58:50.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.446+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3203873701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6819dbd0 con 0x7f4c681016c0 2024-03-24T02:58:50.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.446+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3203873701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c681089e0 con 0x7f4c681016c0 2024-03-24T02:58:50.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.448+0000 7f4c657fa640 1 -- 172.21.15.17:0/3203873701 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4c5800f9c0 con 0x7f4c681016c0 2024-03-24T02:58:50.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.448+0000 7f4c657fa640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4c500a2800 0x7f4c500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:50.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.448+0000 7f4c677fe640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4c500a2800 0x7f4c500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:50.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.448+0000 7f4c657fa640 1 -- 172.21.15.17:0/3203873701 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4c580d2990 con 0x7f4c681016c0 2024-03-24T02:58:50.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.449+0000 7f4c677fe640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4c500a2800 0x7f4c500a4cc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c036040 tx=0x7f4c5c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:50.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.452+0000 7f4c657fa640 1 -- 172.21.15.17:0/3203873701 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4c58014070 con 0x7f4c681016c0 2024-03-24T02:58:50.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.635+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3203873701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c680622d0 con 0x7f4c681016c0 2024-03-24T02:58:50.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.636+0000 7f4c657fa640 1 -- 172.21.15.17:0/3203873701 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24100 (secure 0 0 0) 0x7f4c5809f7f0 con 0x7f4c681016c0 2024-03-24T02:58:50.636 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:50.636 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:58:41.346962+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:58:50.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.639+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3203873701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4c500a2800 msgr2=0x7f4c500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.639+0000 7f4c6e84b640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4c500a2800 0x7f4c500a4cc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c036040 tx=0x7f4c5c000bc0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.639+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3203873701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c681016c0 msgr2=0x7f4c68195770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.639+0000 7f4c6e84b640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c681016c0 0x7f4c68195770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4c58011a60 tx=0x7f4c58011f20 comp rx=0 tx=0).stop 2024-03-24T02:58:50.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.639+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3203873701 shutdown_connections 2024-03-24T02:58:50.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.639+0000 7f4c6e84b640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4c500a2800 0x7f4c500a4cc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.639+0000 7f4c6e84b640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c68103270 0x7f4c6819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.639+0000 7f4c6e84b640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c681028b0 0x7f4c68195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.639+0000 7f4c6e84b640 1 --2- 172.21.15.17:0/3203873701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c681016c0 0x7f4c68195770 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.639+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3203873701 >> 172.21.15.17:0/3203873701 conn(0x7f4c680fcc50 msgr2=0x7f4c68105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:50.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.639+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3203873701 shutdown_connections 2024-03-24T02:58:50.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.639+0000 7f4c6e84b640 1 -- 172.21.15.17:0/3203873701 wait complete. 2024-03-24T02:58:50.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T02:53:54.207139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-24T02:58:34.785192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-24T02:58:35.679772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-24T02:58:39.221772+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-24T02:58:50.650 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T02:58:50.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.756+0000 7fe89a9f4640 1 -- 172.21.15.17:0/4116975568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe894102170 msgr2=0x7fe894108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.756+0000 7fe89a9f4640 1 --2- 172.21.15.17:0/4116975568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe894102170 0x7fe894108a00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009ca0 tx=0x7fe88c02c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.756+0000 7fe89a9f4640 1 -- 172.21.15.17:0/4116975568 shutdown_connections 2024-03-24T02:58:50.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.756+0000 7fe89a9f4640 1 --2- 172.21.15.17:0/4116975568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe894102170 0x7fe894108a00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.756+0000 7fe89a9f4640 1 --2- 172.21.15.17:0/4116975568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8941017b0 0x7fe894101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.756+0000 7fe89a9f4640 1 --2- 172.21.15.17:0/4116975568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8941005c0 0x7fe8941009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.756+0000 7fe89a9f4640 1 -- 172.21.15.17:0/4116975568 >> 172.21.15.17:0/4116975568 conn(0x7fe8940fbb50 msgr2=0x7fe8940fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:50.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.756+0000 7fe89a9f4640 1 -- 172.21.15.17:0/4116975568 shutdown_connections 2024-03-24T02:58:50.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.756+0000 7fe89a9f4640 1 -- 172.21.15.17:0/4116975568 wait complete. 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.756+0000 7fe89a9f4640 1 Processor -- start 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.756+0000 7fe89a9f4640 1 -- start start 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe89a9f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8941005c0 0x7fe894197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe89a9f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8941017b0 0x7fe894197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe89a9f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe894102170 0x7fe89419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe89a9f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89410c240 con 0x7fe8941017b0 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe89a9f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89410c0c0 con 0x7fe894102170 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe89a9f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89410c3c0 con 0x7fe8941005c0 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe8937fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8941017b0 0x7fe894197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe8937fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8941017b0 0x7fe894197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53072/0 (socket says 172.21.15.17:53072) 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe8937fe640 1 -- 172.21.15.17:0/2563322597 learned_addr learned my addr 172.21.15.17:0/2563322597 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe893fff640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8941005c0 0x7fe894197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:50.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe898f91640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe894102170 0x7fe89419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:50.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe8937fe640 1 -- 172.21.15.17:0/2563322597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8941005c0 msgr2=0x7fe894197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe8937fe640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8941005c0 0x7fe894197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe8937fe640 1 -- 172.21.15.17:0/2563322597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe894102170 msgr2=0x7fe89419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe8937fe640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe894102170 0x7fe89419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe8937fe640 1 -- 172.21.15.17:0/2563322597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89419f5a0 con 0x7fe8941017b0 2024-03-24T02:58:50.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe8937fe640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8941017b0 0x7fe894197ec0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe88800e9f0 tx=0x7fe88800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:50.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.757+0000 7fe8917fa640 1 -- 172.21.15.17:0/2563322597 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88800cd80 con 0x7fe8941017b0 2024-03-24T02:58:50.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.758+0000 7fe89a9f4640 1 -- 172.21.15.17:0/2563322597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89419f890 con 0x7fe8941017b0 2024-03-24T02:58:50.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.758+0000 7fe8917fa640 1 -- 172.21.15.17:0/2563322597 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe888005500 con 0x7fe8941017b0 2024-03-24T02:58:50.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.758+0000 7fe8917fa640 1 -- 172.21.15.17:0/2563322597 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888010690 con 0x7fe8941017b0 2024-03-24T02:58:50.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.758+0000 7fe89a9f4640 1 -- 172.21.15.17:0/2563322597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89419fd90 con 0x7fe8941017b0 2024-03-24T02:58:50.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.758+0000 7fe89a9f4640 1 -- 172.21.15.17:0/2563322597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8941a0050 con 0x7fe8941017b0 2024-03-24T02:58:50.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.759+0000 7fe8917fa640 1 -- 172.21.15.17:0/2563322597 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe8880046c0 con 0x7fe8941017b0 2024-03-24T02:58:50.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.760+0000 7fe8917fa640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe87c0a2800 0x7fe87c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:50.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.760+0000 7fe893fff640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe87c0a2800 0x7fe87c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:50.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.760+0000 7fe8917fa640 1 -- 172.21.15.17:0/2563322597 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe8880cf120 con 0x7fe8941017b0 2024-03-24T02:58:50.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.761+0000 7fe893fff640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe87c0a2800 0x7fe87c0a4cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe884005fd0 tx=0x7fe88400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:50.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.765+0000 7fe8917fa640 1 -- 172.21.15.17:0/2563322597 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe88809bf80 con 0x7fe8941017b0 2024-03-24T02:58:50.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.950+0000 7fe89a9f4640 1 -- 172.21.15.17:0/2563322597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8941a0220 con 0x7fe8941017b0 2024-03-24T02:58:50.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.951+0000 7fe8917fa640 1 -- 172.21.15.17:0/2563322597 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe88809b880 con 0x7fe8941017b0 2024-03-24T02:58:50.950 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-24T02:58:50.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.953+0000 7fe89a9f4640 1 -- 172.21.15.17:0/2563322597 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe87c0a2800 msgr2=0x7fe87c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.953+0000 7fe89a9f4640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe87c0a2800 0x7fe87c0a4cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe884005fd0 tx=0x7fe88400b410 comp rx=0 tx=0).stop 2024-03-24T02:58:50.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.953+0000 7fe89a9f4640 1 -- 172.21.15.17:0/2563322597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8941017b0 msgr2=0x7fe894197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:50.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.953+0000 7fe89a9f4640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8941017b0 0x7fe894197ec0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe88800e9f0 tx=0x7fe88800edf0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.953+0000 7fe89a9f4640 1 -- 172.21.15.17:0/2563322597 shutdown_connections 2024-03-24T02:58:50.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.953+0000 7fe89a9f4640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe87c0a2800 0x7fe87c0a4cc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.953+0000 7fe89a9f4640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe894102170 0x7fe89419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.953+0000 7fe89a9f4640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8941017b0 0x7fe894197ec0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.953+0000 7fe89a9f4640 1 --2- 172.21.15.17:0/2563322597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8941005c0 0x7fe894197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:50.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.953+0000 7fe89a9f4640 1 -- 172.21.15.17:0/2563322597 >> 172.21.15.17:0/2563322597 conn(0x7fe8940fbb50 msgr2=0x7fe8940fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:50.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.953+0000 7fe89a9f4640 1 -- 172.21.15.17:0/2563322597 shutdown_connections 2024-03-24T02:58:50.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:50.953+0000 7fe89a9f4640 1 -- 172.21.15.17:0/2563322597 wait complete. 2024-03-24T02:58:50.963 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-24T02:58:51.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.071+0000 7f83a81a9640 1 -- 172.21.15.17:0/598541529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83a01028b0 msgr2=0x7f83a0102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.071+0000 7f83a81a9640 1 --2- 172.21.15.17:0/598541529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83a01028b0 0x7f83a0102d30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8394009900 tx=0x7f839402bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:51.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.071+0000 7f83a81a9640 1 -- 172.21.15.17:0/598541529 shutdown_connections 2024-03-24T02:58:51.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.071+0000 7f83a81a9640 1 --2- 172.21.15.17:0/598541529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83a0103270 0x7f83a0109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.071+0000 7f83a81a9640 1 --2- 172.21.15.17:0/598541529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83a01028b0 0x7f83a0102d30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.071+0000 7f83a81a9640 1 --2- 172.21.15.17:0/598541529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a01016c0 0x7f83a0101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.071+0000 7f83a81a9640 1 -- 172.21.15.17:0/598541529 >> 172.21.15.17:0/598541529 conn(0x7f83a00fcc10 msgr2=0x7f83a00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.071+0000 7f83a81a9640 1 -- 172.21.15.17:0/598541529 shutdown_connections 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.071+0000 7f83a81a9640 1 -- 172.21.15.17:0/598541529 wait complete. 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.071+0000 7f83a81a9640 1 Processor -- start 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.071+0000 7f83a81a9640 1 -- start start 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a81a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a01016c0 0x7f83a0197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a81a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83a01028b0 0x7f83a0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a81a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83a0103270 0x7f83a019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a81a9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a010d340 con 0x7f83a01028b0 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a81a9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a010d1c0 con 0x7f83a01016c0 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a81a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a010d4c0 con 0x7f83a0103270 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a5744640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83a01028b0 0x7f83a0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a5744640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83a01028b0 0x7f83a0197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53090/0 (socket says 172.21.15.17:53090) 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a5744640 1 -- 172.21.15.17:0/3436697242 learned_addr learned my addr 172.21.15.17:0/3436697242 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a6746640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83a0103270 0x7f83a019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a5744640 1 -- 172.21.15.17:0/3436697242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83a0103270 msgr2=0x7f83a019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a5744640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83a0103270 0x7f83a019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a5744640 1 -- 172.21.15.17:0/3436697242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a01016c0 msgr2=0x7f83a0197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a5744640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a01016c0 0x7f83a0197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a5744640 1 -- 172.21.15.17:0/3436697242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a019f5b0 con 0x7f83a01028b0 2024-03-24T02:58:51.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a6746640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83a0103270 0x7f83a019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:51.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a5744640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83a01028b0 0x7f83a0197ed0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8394009230 tx=0x7f8394005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:51.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f839effd640 1 -- 172.21.15.17:0/3436697242 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8394041070 con 0x7f83a01028b0 2024-03-24T02:58:51.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f839effd640 1 -- 172.21.15.17:0/3436697242 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83940044d0 con 0x7f83a01028b0 2024-03-24T02:58:51.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a81a9640 1 -- 172.21.15.17:0/3436697242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a019f840 con 0x7f83a01028b0 2024-03-24T02:58:51.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f839effd640 1 -- 172.21.15.17:0/3436697242 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8394034440 con 0x7f83a01028b0 2024-03-24T02:58:51.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.072+0000 7f83a81a9640 1 -- 172.21.15.17:0/3436697242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a019fd30 con 0x7f83a01028b0 2024-03-24T02:58:51.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.073+0000 7f83a81a9640 1 -- 172.21.15.17:0/3436697242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a01089e0 con 0x7f83a01028b0 2024-03-24T02:58:51.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.074+0000 7f839effd640 1 -- 172.21.15.17:0/3436697242 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8394044070 con 0x7f83a01028b0 2024-03-24T02:58:51.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.075+0000 7f839effd640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f83880a2800 0x7f83880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:51.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.075+0000 7f83a5f45640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f83880a2800 0x7f83880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:51.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.075+0000 7f839effd640 1 -- 172.21.15.17:0/3436697242 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f83940f0a30 con 0x7f83a01028b0 2024-03-24T02:58:51.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.075+0000 7f83a5f45640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f83880a2800 0x7f83880a4cc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8398005fd0 tx=0x7f839800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:51.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.079+0000 7f839effd640 1 -- 172.21.15.17:0/3436697242 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f83940bd830 con 0x7f83a01028b0 2024-03-24T02:58:51.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.260+0000 7f83a81a9640 1 -- 172.21.15.17:0/3436697242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83a00622a0 con 0x7f83a01028b0 2024-03-24T02:58:51.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.261+0000 7f839effd640 1 -- 172.21.15.17:0/3436697242 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f83940bd130 con 0x7f83a01028b0 2024-03-24T02:58:51.260 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T02:58:51.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.263+0000 7f83a81a9640 1 -- 172.21.15.17:0/3436697242 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f83880a2800 msgr2=0x7f83880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.263+0000 7f83a81a9640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f83880a2800 0x7f83880a4cc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8398005fd0 tx=0x7f839800b410 comp rx=0 tx=0).stop 2024-03-24T02:58:51.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.263+0000 7f83a81a9640 1 -- 172.21.15.17:0/3436697242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83a01028b0 msgr2=0x7f83a0197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.263+0000 7f83a81a9640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83a01028b0 0x7f83a0197ed0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8394009230 tx=0x7f8394005d50 comp rx=0 tx=0).stop 2024-03-24T02:58:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.263+0000 7f83a81a9640 1 -- 172.21.15.17:0/3436697242 shutdown_connections 2024-03-24T02:58:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.263+0000 7f83a81a9640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f83880a2800 0x7f83880a4cc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.263+0000 7f83a81a9640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83a0103270 0x7f83a019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.263+0000 7f83a81a9640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83a01028b0 0x7f83a0197ed0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.263+0000 7f83a81a9640 1 --2- 172.21.15.17:0/3436697242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a01016c0 0x7f83a0197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.263+0000 7f83a81a9640 1 -- 172.21.15.17:0/3436697242 >> 172.21.15.17:0/3436697242 conn(0x7f83a00fcc10 msgr2=0x7f83a0105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.263+0000 7f83a81a9640 1 -- 172.21.15.17:0/3436697242 shutdown_connections 2024-03-24T02:58:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.263+0000 7f83a81a9640 1 -- 172.21.15.17:0/3436697242 wait complete. 2024-03-24T02:58:51.272 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:51 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1897484316' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T02:58:51.272 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:51 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3203873701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:51.272 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:51 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2563322597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:58:51.273 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-24T02:58:51.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:51 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1897484316' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T02:58:51.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:51 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3203873701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:51.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:51 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2563322597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:58:51.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:51 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1897484316' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T02:58:51.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:51 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3203873701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:51.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:51 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2563322597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T02:58:51.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 -- 172.21.15.17:0/2267647646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb748102150 msgr2=0x7fb7481089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 --2- 172.21.15.17:0/2267647646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb748102150 0x7fb7481089e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb740009ca0 tx=0x7fb74002c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 -- 172.21.15.17:0/2267647646 shutdown_connections 2024-03-24T02:58:51.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 --2- 172.21.15.17:0/2267647646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb748102150 0x7fb7481089e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 --2- 172.21.15.17:0/2267647646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb748101790 0x7fb748101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 --2- 172.21.15.17:0/2267647646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7481005a0 0x7fb7481009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 -- 172.21.15.17:0/2267647646 >> 172.21.15.17:0/2267647646 conn(0x7fb7480fbb30 msgr2=0x7fb7480fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 -- 172.21.15.17:0/2267647646 shutdown_connections 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 -- 172.21.15.17:0/2267647646 wait complete. 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 Processor -- start 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 -- start start 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7481005a0 0x7fb748199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.379+0000 7fb74f868640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb748101790 0x7fb74819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74f868640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb748102150 0x7fb7481a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74f868640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74810c200 con 0x7fb7481005a0 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74f868640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74810c080 con 0x7fb748102150 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74f868640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74810c380 con 0x7fb748101790 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74d604640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7481005a0 0x7fb748199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74d604640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7481005a0 0x7fb748199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53104/0 (socket says 172.21.15.17:53104) 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74d604640 1 -- 172.21.15.17:0/1962419275 learned_addr learned my addr 172.21.15.17:0/1962419275 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74ce03640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb748101790 0x7fb74819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74d604640 1 -- 172.21.15.17:0/1962419275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb748101790 msgr2=0x7fb74819a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74d604640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb748101790 0x7fb74819a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74de05640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb748102150 0x7fb7481a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74d604640 1 -- 172.21.15.17:0/1962419275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb748102150 msgr2=0x7fb7481a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74d604640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb748102150 0x7fb7481a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74d604640 1 -- 172.21.15.17:0/1962419275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7481a17f0 con 0x7fb7481005a0 2024-03-24T02:58:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74de05640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb748102150 0x7fb7481a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:58:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74d604640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7481005a0 0x7fb748199bd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb74400e9c0 tx=0x7fb74400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb73e7fc640 1 -- 172.21.15.17:0/1962419275 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74400cd80 con 0x7fb7481005a0 2024-03-24T02:58:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb73e7fc640 1 -- 172.21.15.17:0/1962419275 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7440035f0 con 0x7fb7481005a0 2024-03-24T02:58:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74f868640 1 -- 172.21.15.17:0/1962419275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7481a1ae0 con 0x7fb7481005a0 2024-03-24T02:58:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb73e7fc640 1 -- 172.21.15.17:0/1962419275 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744010690 con 0x7fb7481005a0 2024-03-24T02:58:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.380+0000 7fb74f868640 1 -- 172.21.15.17:0/1962419275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7481a2030 con 0x7fb7481005a0 2024-03-24T02:58:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.381+0000 7fb74f868640 1 -- 172.21.15.17:0/1962419275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7481a2200 con 0x7fb7481005a0 2024-03-24T02:58:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.382+0000 7fb73e7fc640 1 -- 172.21.15.17:0/1962419275 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb7440037a0 con 0x7fb7481005a0 2024-03-24T02:58:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.383+0000 7fb73e7fc640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb7340a2800 0x7fb7340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.383+0000 7fb74ce03640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb7340a2800 0x7fb7340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.383+0000 7fb73e7fc640 1 -- 172.21.15.17:0/1962419275 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb7440cf100 con 0x7fb7481005a0 2024-03-24T02:58:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.384+0000 7fb74ce03640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb7340a2800 0x7fb7340a4cc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb74819b0e0 tx=0x7fb73800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:51.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.388+0000 7fb73e7fc640 1 -- 172.21.15.17:0/1962419275 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb74409bf60 con 0x7fb7481005a0 2024-03-24T02:58:51.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.571+0000 7fb74f868640 1 -- 172.21.15.17:0/1962419275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7481a2b40 con 0x7fb7481005a0 2024-03-24T02:58:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.572+0000 7fb73e7fc640 1 -- 172.21.15.17:0/1962419275 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb74409b860 con 0x7fb7481005a0 2024-03-24T02:58:51.571 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-24T02:58:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.573+0000 7fb74f868640 1 -- 172.21.15.17:0/1962419275 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb7340a2800 msgr2=0x7fb7340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.573+0000 7fb74f868640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb7340a2800 0x7fb7340a4cc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb74819b0e0 tx=0x7fb73800a2d0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.573+0000 7fb74f868640 1 -- 172.21.15.17:0/1962419275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7481005a0 msgr2=0x7fb748199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.573+0000 7fb74f868640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7481005a0 0x7fb748199bd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb74400e9c0 tx=0x7fb74400ee80 comp rx=0 tx=0).stop 2024-03-24T02:58:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.574+0000 7fb74f868640 1 -- 172.21.15.17:0/1962419275 shutdown_connections 2024-03-24T02:58:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.574+0000 7fb74f868640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb7340a2800 0x7fb7340a4cc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.574+0000 7fb74f868640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb748102150 0x7fb7481a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.574+0000 7fb74f868640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb748101790 0x7fb74819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.574+0000 7fb74f868640 1 --2- 172.21.15.17:0/1962419275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7481005a0 0x7fb748199bd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.574+0000 7fb74f868640 1 -- 172.21.15.17:0/1962419275 >> 172.21.15.17:0/1962419275 conn(0x7fb7480fbb30 msgr2=0x7fb7480fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:51.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.574+0000 7fb74f868640 1 -- 172.21.15.17:0/1962419275 shutdown_connections 2024-03-24T02:58:51.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.574+0000 7fb74f868640 1 -- 172.21.15.17:0/1962419275 wait complete. 2024-03-24T02:58:51.584 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-24T02:58:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.690+0000 7fc48a6ac640 1 -- 172.21.15.17:0/1069812839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4841005c0 msgr2=0x7fc4841009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.690+0000 7fc48a6ac640 1 --2- 172.21.15.17:0/1069812839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4841005c0 0x7fc4841009e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc474009900 tx=0x7fc47402bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.690+0000 7fc48a6ac640 1 -- 172.21.15.17:0/1069812839 shutdown_connections 2024-03-24T02:58:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.690+0000 7fc48a6ac640 1 --2- 172.21.15.17:0/1069812839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc484102170 0x7fc484108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.690+0000 7fc48a6ac640 1 --2- 172.21.15.17:0/1069812839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4841017b0 0x7fc484101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.690+0000 7fc48a6ac640 1 --2- 172.21.15.17:0/1069812839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4841005c0 0x7fc4841009e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.690+0000 7fc48a6ac640 1 -- 172.21.15.17:0/1069812839 >> 172.21.15.17:0/1069812839 conn(0x7fc4840fbb50 msgr2=0x7fc4840fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.690+0000 7fc48a6ac640 1 -- 172.21.15.17:0/1069812839 shutdown_connections 2024-03-24T02:58:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.690+0000 7fc48a6ac640 1 -- 172.21.15.17:0/1069812839 wait complete. 2024-03-24T02:58:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc48a6ac640 1 Processor -- start 2024-03-24T02:58:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc48a6ac640 1 -- start start 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc48a6ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4841005c0 0x7fc4841968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc48a6ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4841017b0 0x7fc484196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc48a6ac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc484102170 0x7fc48419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc48a6ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48410c210 con 0x7fc4841005c0 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc48a6ac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48410c090 con 0x7fc4841017b0 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc48a6ac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48410c390 con 0x7fc484102170 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc483fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4841005c0 0x7fc4841968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc483fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4841005c0 0x7fc4841968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53120/0 (socket says 172.21.15.17:53120) 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc483fff640 1 -- 172.21.15.17:0/2363370844 learned_addr learned my addr 172.21.15.17:0/2363370844 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc488c49640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc484102170 0x7fc48419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc4837fe640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4841017b0 0x7fc484196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc483fff640 1 -- 172.21.15.17:0/2363370844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc484102170 msgr2=0x7fc48419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc483fff640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc484102170 0x7fc48419de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc483fff640 1 -- 172.21.15.17:0/2363370844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4841017b0 msgr2=0x7fc484196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc483fff640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4841017b0 0x7fc484196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc483fff640 1 -- 172.21.15.17:0/2363370844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48419e500 con 0x7fc4841005c0 2024-03-24T02:58:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.691+0000 7fc483fff640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4841005c0 0x7fc4841968e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc474000c00 tx=0x7fc474005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.692+0000 7fc4817fa640 1 -- 172.21.15.17:0/2363370844 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474041070 con 0x7fc4841005c0 2024-03-24T02:58:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.692+0000 7fc4817fa640 1 -- 172.21.15.17:0/2363370844 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4740044d0 con 0x7fc4841005c0 2024-03-24T02:58:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.692+0000 7fc48a6ac640 1 -- 172.21.15.17:0/2363370844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48419e790 con 0x7fc4841005c0 2024-03-24T02:58:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.692+0000 7fc4817fa640 1 -- 172.21.15.17:0/2363370844 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474034440 con 0x7fc4841005c0 2024-03-24T02:58:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.692+0000 7fc48a6ac640 1 -- 172.21.15.17:0/2363370844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48419ec80 con 0x7fc4841005c0 2024-03-24T02:58:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.693+0000 7fc48a6ac640 1 -- 172.21.15.17:0/2363370844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc484107960 con 0x7fc4841005c0 2024-03-24T02:58:51.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.694+0000 7fc4817fa640 1 -- 172.21.15.17:0/2363370844 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc474044070 con 0x7fc4841005c0 2024-03-24T02:58:51.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.694+0000 7fc4817fa640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc46c0a2800 0x7fc46c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:51.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.694+0000 7fc4817fa640 1 -- 172.21.15.17:0/2363370844 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc4740f0a60 con 0x7fc4841005c0 2024-03-24T02:58:51.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.694+0000 7fc4837fe640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc46c0a2800 0x7fc46c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:51.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.695+0000 7fc4837fe640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc46c0a2800 0x7fc46c0a4cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc4780095f0 tx=0x7fc478005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.698+0000 7fc4817fa640 1 -- 172.21.15.17:0/2363370844 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc4740bd8c0 con 0x7fc4841005c0 2024-03-24T02:58:51.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.881+0000 7fc48a6ac640 1 -- 172.21.15.17:0/2363370844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc48402d200 con 0x7fc4841005c0 2024-03-24T02:58:51.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.882+0000 7fc4817fa640 1 -- 172.21.15.17:0/2363370844 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc4740bd1c0 con 0x7fc4841005c0 2024-03-24T02:58:51.881 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T02:58:51.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.884+0000 7fc48a6ac640 1 -- 172.21.15.17:0/2363370844 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc46c0a2800 msgr2=0x7fc46c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.884+0000 7fc48a6ac640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc46c0a2800 0x7fc46c0a4cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc4780095f0 tx=0x7fc478005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:51.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.884+0000 7fc48a6ac640 1 -- 172.21.15.17:0/2363370844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4841005c0 msgr2=0x7fc4841968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:51.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.884+0000 7fc48a6ac640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4841005c0 0x7fc4841968e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc474000c00 tx=0x7fc474005d50 comp rx=0 tx=0).stop 2024-03-24T02:58:51.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.884+0000 7fc48a6ac640 1 -- 172.21.15.17:0/2363370844 shutdown_connections 2024-03-24T02:58:51.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.884+0000 7fc48a6ac640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc46c0a2800 0x7fc46c0a4cc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.884+0000 7fc48a6ac640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc484102170 0x7fc48419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.884+0000 7fc48a6ac640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4841017b0 0x7fc484196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.884+0000 7fc48a6ac640 1 --2- 172.21.15.17:0/2363370844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4841005c0 0x7fc4841968e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.884+0000 7fc48a6ac640 1 -- 172.21.15.17:0/2363370844 >> 172.21.15.17:0/2363370844 conn(0x7fc4840fbb50 msgr2=0x7fc484104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.884+0000 7fc48a6ac640 1 -- 172.21.15.17:0/2363370844 shutdown_connections 2024-03-24T02:58:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.884+0000 7fc48a6ac640 1 -- 172.21.15.17:0/2363370844 wait complete. 2024-03-24T02:58:51.894 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-24T02:58:52.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.998+0000 7ff097f22640 1 -- 172.21.15.17:0/1105112598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0901016b0 msgr2=0x7ff090101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.998+0000 7ff097f22640 1 --2- 172.21.15.17:0/1105112598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0901016b0 0x7ff090101ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009900 tx=0x7ff08c02bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.998+0000 7ff097f22640 1 -- 172.21.15.17:0/1105112598 shutdown_connections 2024-03-24T02:58:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.998+0000 7ff097f22640 1 --2- 172.21.15.17:0/1105112598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff090103260 0x7ff090109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.998+0000 7ff097f22640 1 --2- 172.21.15.17:0/1105112598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0901028a0 0x7ff090102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.998+0000 7ff097f22640 1 --2- 172.21.15.17:0/1105112598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0901016b0 0x7ff090101ad0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.998+0000 7ff097f22640 1 -- 172.21.15.17:0/1105112598 >> 172.21.15.17:0/1105112598 conn(0x7ff0900fcc40 msgr2=0x7ff0900ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.998+0000 7ff097f22640 1 -- 172.21.15.17:0/1105112598 shutdown_connections 2024-03-24T02:58:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.998+0000 7ff097f22640 1 -- 172.21.15.17:0/1105112598 wait complete. 2024-03-24T02:58:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff097f22640 1 Processor -- start 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff097f22640 1 -- start start 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff097f22640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0901016b0 0x7ff090197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff097f22640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0901028a0 0x7ff090197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff097f22640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff090103260 0x7ff09019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff097f22640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09010d380 con 0x7ff0901016b0 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff097f22640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09010d200 con 0x7ff090103260 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff097f22640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09010d500 con 0x7ff0901028a0 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff095cbe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0901016b0 0x7ff090197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff095cbe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0901016b0 0x7ff090197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53138/0 (socket says 172.21.15.17:53138) 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff095cbe640 1 -- 172.21.15.17:0/4022521060 learned_addr learned my addr 172.21.15.17:0/4022521060 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff0954bd640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0901028a0 0x7ff090197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff0964bf640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff090103260 0x7ff09019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff095cbe640 1 -- 172.21.15.17:0/4022521060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0901028a0 msgr2=0x7ff090197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff095cbe640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0901028a0 0x7ff090197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff095cbe640 1 -- 172.21.15.17:0/4022521060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff090103260 msgr2=0x7ff09019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff095cbe640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff090103260 0x7ff09019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff095cbe640 1 -- 172.21.15.17:0/4022521060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09019f610 con 0x7ff0901016b0 2024-03-24T02:58:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:51.999+0000 7ff0964bf640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff090103260 0x7ff09019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:58:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.000+0000 7ff095cbe640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0901016b0 0x7ff090197a90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff08c005ab0 tx=0x7ff08c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.000+0000 7ff086ffd640 1 -- 172.21.15.17:0/4022521060 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c043020 con 0x7ff0901016b0 2024-03-24T02:58:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.000+0000 7ff086ffd640 1 -- 172.21.15.17:0/4022521060 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff08c0054e0 con 0x7ff0901016b0 2024-03-24T02:58:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.000+0000 7ff097f22640 1 -- 172.21.15.17:0/4022521060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09019f8a0 con 0x7ff0901016b0 2024-03-24T02:58:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.000+0000 7ff086ffd640 1 -- 172.21.15.17:0/4022521060 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c003c70 con 0x7ff0901016b0 2024-03-24T02:58:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.000+0000 7ff097f22640 1 -- 172.21.15.17:0/4022521060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09019fdf0 con 0x7ff0901016b0 2024-03-24T02:58:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.001+0000 7ff097f22640 1 -- 172.21.15.17:0/4022521060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff090108970 con 0x7ff0901016b0 2024-03-24T02:58:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.002+0000 7ff086ffd640 1 -- 172.21.15.17:0/4022521060 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff08c046020 con 0x7ff0901016b0 2024-03-24T02:58:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.002+0000 7ff086ffd640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff07c0a2800 0x7ff07c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.002+0000 7ff0954bd640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff07c0a2800 0x7ff07c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.002+0000 7ff086ffd640 1 -- 172.21.15.17:0/4022521060 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff08c0f0e50 con 0x7ff0901016b0 2024-03-24T02:58:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.003+0000 7ff0954bd640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff07c0a2800 0x7ff07c0a4cc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff090198f00 tx=0x7ff08000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:52.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.007+0000 7ff086ffd640 1 -- 172.21.15.17:0/4022521060 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff08c04d020 con 0x7ff0901016b0 2024-03-24T02:58:52.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.203+0000 7ff097f22640 1 -- 172.21.15.17:0/4022521060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff09002d2b0 con 0x7ff0901016b0 2024-03-24T02:58:52.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.204+0000 7ff086ffd640 1 -- 172.21.15.17:0/4022521060 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+586 (secure 0 0 0) 0x7ff08c0bdcb0 con 0x7ff0901016b0 2024-03-24T02:58:52.203 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/1870031079 2024-03-25T02:51:29.864563+0000 2024-03-24T02:58:52.203 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/3007657427 2024-03-25T02:51:29.864563+0000 2024-03-24T02:58:52.203 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/3007657427 2024-03-25T02:51:29.864563+0000 2024-03-24T02:58:52.204 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/2226886960 2024-03-25T02:51:02.292414+0000 2024-03-24T02:58:52.204 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/4291784529 2024-03-25T02:51:02.292414+0000 2024-03-24T02:58:52.204 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/4202143244 2024-03-25T02:51:02.292414+0000 2024-03-24T02:58:52.204 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/4291784529 2024-03-25T02:51:02.292414+0000 2024-03-24T02:58:52.204 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/630595937 2024-03-25T02:51:02.292414+0000 2024-03-24T02:58:52.204 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/532405 2024-03-25T02:51:29.864563+0000 2024-03-24T02:58:52.204 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/467902990 2024-03-25T02:51:29.864563+0000 2024-03-24T02:58:52.204 INFO:teuthology.orchestra.run.smithi017.stderr:listed 10 entries 2024-03-24T02:58:52.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.206+0000 7ff097f22640 1 -- 172.21.15.17:0/4022521060 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff07c0a2800 msgr2=0x7ff07c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.206+0000 7ff097f22640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff07c0a2800 0x7ff07c0a4cc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff090198f00 tx=0x7ff08000a2d0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.206+0000 7ff097f22640 1 -- 172.21.15.17:0/4022521060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0901016b0 msgr2=0x7ff090197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.206+0000 7ff097f22640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0901016b0 0x7ff090197a90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff08c005ab0 tx=0x7ff08c02c6c0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.206+0000 7ff097f22640 1 -- 172.21.15.17:0/4022521060 shutdown_connections 2024-03-24T02:58:52.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.206+0000 7ff097f22640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff07c0a2800 0x7ff07c0a4cc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.206+0000 7ff097f22640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff090103260 0x7ff09019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.206+0000 7ff097f22640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0901028a0 0x7ff090197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.206+0000 7ff097f22640 1 --2- 172.21.15.17:0/4022521060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0901016b0 0x7ff090197a90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.206+0000 7ff097f22640 1 -- 172.21.15.17:0/4022521060 >> 172.21.15.17:0/4022521060 conn(0x7ff0900fcc40 msgr2=0x7ff090105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:52.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.206+0000 7ff097f22640 1 -- 172.21.15.17:0/4022521060 shutdown_connections 2024-03-24T02:58:52.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.206+0000 7ff097f22640 1 -- 172.21.15.17:0/4022521060 wait complete. 2024-03-24T02:58:52.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:52 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3436697242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T02:58:52.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:52 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1962419275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T02:58:52.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:52 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2363370844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T02:58:52.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:52 smithi017 ceph-mon[31926]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:52.216 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi017.front.sepia.ceph.com... 2024-03-24T02:58:52.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:52.216 DEBUG:teuthology.orchestra.run.smithi017:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-24T02:58:52.241 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[76062]: fuse finished with error 0 and tester_r 0 2024-03-24T02:58:52.243 INFO:teuthology.orchestra.run:waiting for 300 2024-03-24T02:58:52.260 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi017.front.sepia.ceph.com 2024-03-24T02:58:52.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T02:58:52.261 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-24T02:58:52.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:52 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3436697242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T02:58:52.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:52 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1962419275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T02:58:52.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:52 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2363370844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T02:58:52.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:52 smithi188 ceph-mon[36206]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:52.319 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T02:58:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:52 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3436697242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T02:58:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:52 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1962419275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T02:58:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:52 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2363370844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T02:58:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:52 smithi019 ceph-mon[35944]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T02:58:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.463+0000 7f7cc0f53640 1 -- 172.21.15.17:0/2066144492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cbc1017d0 msgr2=0x7f7cbc101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.463+0000 7f7cc0f53640 1 --2- 172.21.15.17:0/2066144492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cbc1017d0 0x7f7cbc101c50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009900 tx=0x7f7ca402bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.463+0000 7f7cc0f53640 1 -- 172.21.15.17:0/2066144492 shutdown_connections 2024-03-24T02:58:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.463+0000 7f7cc0f53640 1 --2- 172.21.15.17:0/2066144492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cbc102190 0x7f7cbc108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.463+0000 7f7cc0f53640 1 --2- 172.21.15.17:0/2066144492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cbc1017d0 0x7f7cbc101c50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.463+0000 7f7cc0f53640 1 --2- 172.21.15.17:0/2066144492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cbc1005e0 0x7f7cbc100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.463+0000 7f7cc0f53640 1 -- 172.21.15.17:0/2066144492 >> 172.21.15.17:0/2066144492 conn(0x7f7cbc0fbb50 msgr2=0x7f7cbc0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.463+0000 7f7cc0f53640 1 -- 172.21.15.17:0/2066144492 shutdown_connections 2024-03-24T02:58:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.463+0000 7f7cc0f53640 1 -- 172.21.15.17:0/2066144492 wait complete. 2024-03-24T02:58:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.463+0000 7f7cc0f53640 1 Processor -- start 2024-03-24T02:58:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cc0f53640 1 -- start start 2024-03-24T02:58:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cc0f53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cbc1005e0 0x7f7cbc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cc0f53640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cbc1017d0 0x7f7cbc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cc0f53640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cbc102190 0x7f7cbc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cc0f53640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc10c2e0 con 0x7f7cbc1017d0 2024-03-24T02:58:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cc0f53640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc10c160 con 0x7f7cbc1005e0 2024-03-24T02:58:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cc0f53640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc10c460 con 0x7f7cbc102190 2024-03-24T02:58:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cb3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cbc1017d0 0x7f7cbc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cb3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cbc1017d0 0x7f7cbc197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53160/0 (socket says 172.21.15.17:53160) 2024-03-24T02:58:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cb3fff640 1 -- 172.21.15.17:0/1457591 learned_addr learned my addr 172.21.15.17:0/1457591 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cbad9d640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cbc102190 0x7f7cbc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cba59c640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cbc1005e0 0x7f7cbc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cb3fff640 1 -- 172.21.15.17:0/1457591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cbc102190 msgr2=0x7f7cbc19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cb3fff640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cbc102190 0x7f7cbc19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cb3fff640 1 -- 172.21.15.17:0/1457591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cbc1005e0 msgr2=0x7f7cbc197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cb3fff640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cbc1005e0 0x7f7cbc197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cb3fff640 1 -- 172.21.15.17:0/1457591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc19f660 con 0x7f7cbc1017d0 2024-03-24T02:58:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cba59c640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cbc1005e0 0x7f7cbc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:58:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cb3fff640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cbc1017d0 0x7f7cbc197f80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009230 tx=0x7f7ca4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cb37fe640 1 -- 172.21.15.17:0/1457591 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4041070 con 0x7f7cbc1017d0 2024-03-24T02:58:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cc0f53640 1 -- 172.21.15.17:0/1457591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cbc19f8f0 con 0x7f7cbc1017d0 2024-03-24T02:58:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cb37fe640 1 -- 172.21.15.17:0/1457591 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ca40044d0 con 0x7f7cbc1017d0 2024-03-24T02:58:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.464+0000 7f7cb37fe640 1 -- 172.21.15.17:0/1457591 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4034440 con 0x7f7cbc1017d0 2024-03-24T02:58:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.465+0000 7f7cc0f53640 1 -- 172.21.15.17:0/1457591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cbc19fe30 con 0x7f7cbc1017d0 2024-03-24T02:58:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.465+0000 7f7cc0f53640 1 -- 172.21.15.17:0/1457591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cbc1a00c0 con 0x7f7cbc1017d0 2024-03-24T02:58:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.466+0000 7f7cb37fe640 1 -- 172.21.15.17:0/1457591 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7ca4044070 con 0x7f7cbc1017d0 2024-03-24T02:58:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.467+0000 7f7cb37fe640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7ca80a2800 0x7f7ca80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.467+0000 7f7cb37fe640 1 -- 172.21.15.17:0/1457591 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f7ca40f0980 con 0x7f7cbc1017d0 2024-03-24T02:58:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.467+0000 7f7cba59c640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7ca80a2800 0x7f7ca80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.468+0000 7f7cba59c640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7ca80a2800 0x7f7ca80a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc101630 tx=0x7f7cac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:52.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.472+0000 7f7cb37fe640 1 -- 172.21.15.17:0/1457591 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ca40bd7e0 con 0x7f7cbc1017d0 2024-03-24T02:58:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.657+0000 7f7cc0f53640 1 -- 172.21.15.17:0/1457591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cbc1a0780 con 0x7f7cbc1017d0 2024-03-24T02:58:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.658+0000 7f7cb37fe640 1 -- 172.21.15.17:0/1457591 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24100 (secure 0 0 0) 0x7f7ca40bd0e0 con 0x7f7cbc1017d0 2024-03-24T02:58:52.658 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:52.658 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:58:41.346962+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:58:52.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.660+0000 7f7cc0f53640 1 -- 172.21.15.17:0/1457591 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7ca80a2800 msgr2=0x7f7ca80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.660+0000 7f7cc0f53640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7ca80a2800 0x7f7ca80a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc101630 tx=0x7f7cac005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:52.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.660+0000 7f7cc0f53640 1 -- 172.21.15.17:0/1457591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cbc1017d0 msgr2=0x7f7cbc197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.660+0000 7f7cc0f53640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cbc1017d0 0x7f7cbc197f80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009230 tx=0x7f7ca4005d50 comp rx=0 tx=0).stop 2024-03-24T02:58:52.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.660+0000 7f7cc0f53640 1 -- 172.21.15.17:0/1457591 shutdown_connections 2024-03-24T02:58:52.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.661+0000 7f7cc0f53640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f7ca80a2800 0x7f7ca80a4cc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.661+0000 7f7cc0f53640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cbc102190 0x7f7cbc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.661+0000 7f7cc0f53640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cbc1017d0 0x7f7cbc197f80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.661+0000 7f7cc0f53640 1 --2- 172.21.15.17:0/1457591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cbc1005e0 0x7f7cbc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.661+0000 7f7cc0f53640 1 -- 172.21.15.17:0/1457591 >> 172.21.15.17:0/1457591 conn(0x7f7cbc0fbb50 msgr2=0x7f7cbc0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:52.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.661+0000 7f7cc0f53640 1 -- 172.21.15.17:0/1457591 shutdown_connections 2024-03-24T02:58:52.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.661+0000 7f7cc0f53640 1 -- 172.21.15.17:0/1457591 wait complete. 2024-03-24T02:58:52.672 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T02:58:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.775+0000 7f191e79d640 1 -- 172.21.15.17:0/4269571545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19181073a0 msgr2=0x7f19181097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.775+0000 7f191e79d640 1 --2- 172.21.15.17:0/4269571545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19181073a0 0x7f19181097f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1910009ca0 tx=0x7f191002c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.775+0000 7f191e79d640 1 -- 172.21.15.17:0/4269571545 shutdown_connections 2024-03-24T02:58:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.775+0000 7f191e79d640 1 --2- 172.21.15.17:0/4269571545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19181073a0 0x7f19181097f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.775+0000 7f191e79d640 1 --2- 172.21.15.17:0/4269571545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1918104a70 0x7f1918106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.775+0000 7f191e79d640 1 --2- 172.21.15.17:0/4269571545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1918102140 0x7f1918104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.775+0000 7f191e79d640 1 -- 172.21.15.17:0/4269571545 >> 172.21.15.17:0/4269571545 conn(0x7f19180fbb50 msgr2=0x7f19180fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.775+0000 7f191e79d640 1 -- 172.21.15.17:0/4269571545 shutdown_connections 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.775+0000 7f191e79d640 1 -- 172.21.15.17:0/4269571545 wait complete. 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.776+0000 7f191e79d640 1 Processor -- start 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.776+0000 7f191e79d640 1 -- start start 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.776+0000 7f191e79d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1918102140 0x7f1918101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.776+0000 7f191e79d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1918104a70 0x7f19181000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.776+0000 7f191e79d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19181073a0 0x7f1918100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.776+0000 7f191e79d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191810d340 con 0x7f1918102140 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.776+0000 7f191e79d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191810d1c0 con 0x7f1918104a70 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.776+0000 7f191e79d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191810d4c0 con 0x7f19181073a0 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.776+0000 7f1917fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1918102140 0x7f1918101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.776+0000 7f1917fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1918102140 0x7f1918101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53180/0 (socket says 172.21.15.17:53180) 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.776+0000 7f1917fff640 1 -- 172.21.15.17:0/1824562743 learned_addr learned my addr 172.21.15.17:0/1824562743 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.776+0000 7f191cd3a640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19181073a0 0x7f1918100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:52.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f19177fe640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1918104a70 0x7f19181000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f1917fff640 1 -- 172.21.15.17:0/1824562743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19181073a0 msgr2=0x7f1918100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f1917fff640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19181073a0 0x7f1918100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f1917fff640 1 -- 172.21.15.17:0/1824562743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1918104a70 msgr2=0x7f19181000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f1917fff640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1918104a70 0x7f19181000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f1917fff640 1 -- 172.21.15.17:0/1824562743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1918100ec0 con 0x7f1918102140 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f19177fe640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1918104a70 0x7f19181000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f191cd3a640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19181073a0 0x7f1918100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f1917fff640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1918102140 0x7f1918101ab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f190800e9f0 tx=0x7f190800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f19157fa640 1 -- 172.21.15.17:0/1824562743 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190800cd80 con 0x7f1918102140 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f19157fa640 1 -- 172.21.15.17:0/1824562743 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1908010910 con 0x7f1918102140 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f191e79d640 1 -- 172.21.15.17:0/1824562743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19181a1aa0 con 0x7f1918102140 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f19157fa640 1 -- 172.21.15.17:0/1824562743 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908018990 con 0x7f1918102140 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.777+0000 7f191e79d640 1 -- 172.21.15.17:0/1824562743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19181a1ff0 con 0x7f1918102140 2024-03-24T02:58:52.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.778+0000 7f191e79d640 1 -- 172.21.15.17:0/1824562743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19180fd640 con 0x7f1918102140 2024-03-24T02:58:52.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.779+0000 7f19157fa640 1 -- 172.21.15.17:0/1824562743 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1908010ac0 con 0x7f1918102140 2024-03-24T02:58:52.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.780+0000 7f19157fa640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f19000a2800 0x7f19000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:52.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.780+0000 7f19177fe640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f19000a2800 0x7f19000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:52.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.780+0000 7f19157fa640 1 -- 172.21.15.17:0/1824562743 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f19080cf500 con 0x7f1918102140 2024-03-24T02:58:52.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.780+0000 7f19177fe640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f19000a2800 0x7f19000a4cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1918101820 tx=0x7f190c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.783+0000 7f19157fa640 1 -- 172.21.15.17:0/1824562743 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f190809c360 con 0x7f1918102140 2024-03-24T02:58:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.967+0000 7f191e79d640 1 -- 172.21.15.17:0/1824562743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f191802d2d0 con 0x7f1918102140 2024-03-24T02:58:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.969+0000 7f19157fa640 1 -- 172.21.15.17:0/1824562743 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24100 (secure 0 0 0) 0x7f190809bc60 con 0x7f1918102140 2024-03-24T02:58:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:52.969 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:58:41.346962+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:58:52.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.971+0000 7f191e79d640 1 -- 172.21.15.17:0/1824562743 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f19000a2800 msgr2=0x7f19000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.971+0000 7f191e79d640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f19000a2800 0x7f19000a4cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1918101820 tx=0x7f190c005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:52.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.971+0000 7f191e79d640 1 -- 172.21.15.17:0/1824562743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1918102140 msgr2=0x7f1918101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:52.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.971+0000 7f191e79d640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1918102140 0x7f1918101ab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f190800e9f0 tx=0x7f190800eeb0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.971+0000 7f191e79d640 1 -- 172.21.15.17:0/1824562743 shutdown_connections 2024-03-24T02:58:52.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.971+0000 7f191e79d640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f19000a2800 0x7f19000a4cc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.971+0000 7f191e79d640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19181073a0 0x7f1918100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.971+0000 7f191e79d640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1918104a70 0x7f19181000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.971+0000 7f191e79d640 1 --2- 172.21.15.17:0/1824562743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1918102140 0x7f1918101ab0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:52.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.971+0000 7f191e79d640 1 -- 172.21.15.17:0/1824562743 >> 172.21.15.17:0/1824562743 conn(0x7f19180fbb50 msgr2=0x7f1918102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:52.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.971+0000 7f191e79d640 1 -- 172.21.15.17:0/1824562743 shutdown_connections 2024-03-24T02:58:52.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:52.971+0000 7f191e79d640 1 -- 172.21.15.17:0/1824562743 wait complete. 2024-03-24T02:58:52.984 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-24T02:58:53.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.091+0000 7f50f531b640 1 -- 172.21.15.17:0/870698075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f00ffd20 msgr2=0x7f50f0100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:53.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.091+0000 7f50f531b640 1 --2- 172.21.15.17:0/870698075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f00ffd20 0x7f50f0100140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009900 tx=0x7f50e002bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:53.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.091+0000 7f50f531b640 1 -- 172.21.15.17:0/870698075 shutdown_connections 2024-03-24T02:58:53.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.091+0000 7f50f531b640 1 --2- 172.21.15.17:0/870698075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50f01018d0 0x7f50f0108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:53.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.091+0000 7f50f531b640 1 --2- 172.21.15.17:0/870698075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50f0100f10 0x7f50f0101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:53.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.091+0000 7f50f531b640 1 --2- 172.21.15.17:0/870698075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f00ffd20 0x7f50f0100140 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:53.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.091+0000 7f50f531b640 1 -- 172.21.15.17:0/870698075 >> 172.21.15.17:0/870698075 conn(0x7f50f00fb2b0 msgr2=0x7f50f00fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:53.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.091+0000 7f50f531b640 1 -- 172.21.15.17:0/870698075 shutdown_connections 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.091+0000 7f50f531b640 1 -- 172.21.15.17:0/870698075 wait complete. 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.091+0000 7f50f531b640 1 Processor -- start 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.091+0000 7f50f531b640 1 -- start start 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50f531b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50f00ffd20 0x7f50f0199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50f531b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50f0100f10 0x7f50f019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50f531b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f01018d0 0x7f50f01a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50f531b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f010b130 con 0x7f50f01018d0 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50f531b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f010afb0 con 0x7f50f00ffd20 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50f531b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f010b2b0 con 0x7f50f0100f10 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50ef7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f01018d0 0x7f50f01a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50ef7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f01018d0 0x7f50f01a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53194/0 (socket says 172.21.15.17:53194) 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50ef7fe640 1 -- 172.21.15.17:0/4163792526 learned_addr learned my addr 172.21.15.17:0/4163792526 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50ee7fc640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50f0100f10 0x7f50f019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50eeffd640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50f00ffd20 0x7f50f0199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:53.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50ef7fe640 1 -- 172.21.15.17:0/4163792526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50f0100f10 msgr2=0x7f50f019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:53.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50ef7fe640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50f0100f10 0x7f50f019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:53.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50ef7fe640 1 -- 172.21.15.17:0/4163792526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50f00ffd20 msgr2=0x7f50f0199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:53.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50ef7fe640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50f00ffd20 0x7f50f0199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:53.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50ef7fe640 1 -- 172.21.15.17:0/4163792526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f01a1810 con 0x7f50f01018d0 2024-03-24T02:58:53.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50ee7fc640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50f0100f10 0x7f50f019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:58:53.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50eeffd640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50f00ffd20 0x7f50f0199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:58:53.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50ef7fe640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f01018d0 0x7f50f01a11a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f50e800ec60 tx=0x7f50e800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:53.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50d7fff640 1 -- 172.21.15.17:0/4163792526 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e8004d20 con 0x7f50f01018d0 2024-03-24T02:58:53.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50d7fff640 1 -- 172.21.15.17:0/4163792526 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50e800ba50 con 0x7f50f01018d0 2024-03-24T02:58:53.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50f531b640 1 -- 172.21.15.17:0/4163792526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f01a1b00 con 0x7f50f01018d0 2024-03-24T02:58:53.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50d7fff640 1 -- 172.21.15.17:0/4163792526 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e8005740 con 0x7f50f01018d0 2024-03-24T02:58:53.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.092+0000 7f50f531b640 1 -- 172.21.15.17:0/4163792526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f01a2050 con 0x7f50f01018d0 2024-03-24T02:58:53.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.093+0000 7f50f531b640 1 -- 172.21.15.17:0/4163792526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f0106fe0 con 0x7f50f01018d0 2024-03-24T02:58:53.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.094+0000 7f50d7fff640 1 -- 172.21.15.17:0/4163792526 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f50e80058e0 con 0x7f50f01018d0 2024-03-24T02:58:53.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.095+0000 7f50d7fff640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f50d80a2800 0x7f50d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:53.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.095+0000 7f50eeffd640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f50d80a2800 0x7f50d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:53.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.095+0000 7f50d7fff640 1 -- 172.21.15.17:0/4163792526 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f50e80cf120 con 0x7f50f01018d0 2024-03-24T02:58:53.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.095+0000 7f50eeffd640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f50d80a2800 0x7f50d80a4cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f50e0036040 tx=0x7f50e0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:53.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.099+0000 7f50d7fff640 1 -- 172.21.15.17:0/4163792526 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f50e809bf80 con 0x7f50f01018d0 2024-03-24T02:58:53.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:53 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4022521060' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T02:58:53.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:53 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1457591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:53.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:53 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1824562743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:53.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:53.321+0000 7f50f531b640 1 -- 172.21.15.17:0/4163792526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f50f0062340 con 0x7f50f01018d0 2024-03-24T02:58:53.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:53 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4022521060' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T02:58:53.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:53 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1457591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:53.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:53 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1824562743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:53 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4022521060' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T02:58:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:53 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1457591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:53 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1824562743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:58:54.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.026+0000 7f50d7fff640 1 -- 172.21.15.17:0/4163792526 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f50e809b880 con 0x7f50f01018d0 2024-03-24T02:58:54.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.029+0000 7f50f531b640 1 -- 172.21.15.17:0/4163792526 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f50d80a2800 msgr2=0x7f50d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:54.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.029+0000 7f50f531b640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f50d80a2800 0x7f50d80a4cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f50e0036040 tx=0x7f50e0000bc0 comp rx=0 tx=0).stop 2024-03-24T02:58:54.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.029+0000 7f50f531b640 1 -- 172.21.15.17:0/4163792526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f01018d0 msgr2=0x7f50f01a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:54.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.029+0000 7f50f531b640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f01018d0 0x7f50f01a11a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f50e800ec60 tx=0x7f50e800c7a0 comp rx=0 tx=0).stop 2024-03-24T02:58:54.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.029+0000 7f50f531b640 1 -- 172.21.15.17:0/4163792526 shutdown_connections 2024-03-24T02:58:54.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.029+0000 7f50f531b640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f50d80a2800 0x7f50d80a4cc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:54.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.029+0000 7f50f531b640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f01018d0 0x7f50f01a11a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:54.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.029+0000 7f50f531b640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50f0100f10 0x7f50f019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:54.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.029+0000 7f50f531b640 1 --2- 172.21.15.17:0/4163792526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50f00ffd20 0x7f50f0199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:54.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.029+0000 7f50f531b640 1 -- 172.21.15.17:0/4163792526 >> 172.21.15.17:0/4163792526 conn(0x7f50f00fb2b0 msgr2=0x7f50f01041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:54.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.029+0000 7f50f531b640 1 -- 172.21.15.17:0/4163792526 shutdown_connections 2024-03-24T02:58:54.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.029+0000 7f50f531b640 1 -- 172.21.15.17:0/4163792526 wait complete. 2024-03-24T02:58:54.042 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-24T02:58:54.042 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-24T02:58:54.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.155+0000 7fcbb8cea640 1 -- 172.21.15.17:0/380373290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb40ff960 msgr2=0x7fcbb40ffd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:54.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.155+0000 7fcbb8cea640 1 --2- 172.21.15.17:0/380373290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb40ff960 0x7fcbb40ffd80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009930 tx=0x7fcba402bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.155+0000 7fcbb8cea640 1 -- 172.21.15.17:0/380373290 shutdown_connections 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.155+0000 7fcbb8cea640 1 --2- 172.21.15.17:0/380373290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbb41032f0 0x7fcbb4109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.155+0000 7fcbb8cea640 1 --2- 172.21.15.17:0/380373290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb4102930 0x7fcbb4102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.155+0000 7fcbb8cea640 1 --2- 172.21.15.17:0/380373290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb40ff960 0x7fcbb40ffd80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.155+0000 7fcbb8cea640 1 -- 172.21.15.17:0/380373290 >> 172.21.15.17:0/380373290 conn(0x7fcbb40fb2b0 msgr2=0x7fcbb40fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.155+0000 7fcbb8cea640 1 -- 172.21.15.17:0/380373290 shutdown_connections 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.155+0000 7fcbb8cea640 1 -- 172.21.15.17:0/380373290 wait complete. 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb8cea640 1 Processor -- start 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb8cea640 1 -- start start 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb8cea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40ff960 0x7fcbb41022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb8cea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbb4102930 0x7fcbb41008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb8cea640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb41032f0 0x7fcbb4100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb8cea640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb410d360 con 0x7fcbb41032f0 2024-03-24T02:58:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb8cea640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb410d1e0 con 0x7fcbb40ff960 2024-03-24T02:58:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb8cea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb410d4e0 con 0x7fcbb4102930 2024-03-24T02:58:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb41032f0 0x7fcbb4100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb41032f0 0x7fcbb4100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53206/0 (socket says 172.21.15.17:53206) 2024-03-24T02:58:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb2d9d640 1 -- 172.21.15.17:0/1265360696 learned_addr learned my addr 172.21.15.17:0/1265360696 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb1d9b640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbb4102930 0x7fcbb41008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb259c640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40ff960 0x7fcbb41022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb2d9d640 1 -- 172.21.15.17:0/1265360696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbb4102930 msgr2=0x7fcbb41008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb2d9d640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbb4102930 0x7fcbb41008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb2d9d640 1 -- 172.21.15.17:0/1265360696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40ff960 msgr2=0x7fcbb41022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb2d9d640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40ff960 0x7fcbb41022a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb2d9d640 1 -- 172.21.15.17:0/1265360696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb41016b0 con 0x7fcbb41032f0 2024-03-24T02:58:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.156+0000 7fcbb2d9d640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb41032f0 0x7fcbb4100e20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00eb10 tx=0x7fcbac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:54.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.157+0000 7fcba37fe640 1 -- 172.21.15.17:0/1265360696 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac004b20 con 0x7fcbb41032f0 2024-03-24T02:58:54.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.157+0000 7fcba37fe640 1 -- 172.21.15.17:0/1265360696 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbac004cc0 con 0x7fcbb41032f0 2024-03-24T02:58:54.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.157+0000 7fcba37fe640 1 -- 172.21.15.17:0/1265360696 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac010760 con 0x7fcbb41032f0 2024-03-24T02:58:54.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.157+0000 7fcbb8cea640 1 -- 172.21.15.17:0/1265360696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb41a1af0 con 0x7fcbb41032f0 2024-03-24T02:58:54.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.157+0000 7fcbb8cea640 1 -- 172.21.15.17:0/1265360696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb41a2040 con 0x7fcbb41032f0 2024-03-24T02:58:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.158+0000 7fcbb8cea640 1 -- 172.21.15.17:0/1265360696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb4108a60 con 0x7fcbb41032f0 2024-03-24T02:58:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.159+0000 7fcba37fe640 1 -- 172.21.15.17:0/1265360696 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fcbac005020 con 0x7fcbb41032f0 2024-03-24T02:58:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.159+0000 7fcba37fe640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcb9c0a2880 0x7fcb9c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.160+0000 7fcbb259c640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcb9c0a2880 0x7fcb9c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.160+0000 7fcba37fe640 1 -- 172.21.15.17:0/1265360696 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fcbac0cf2f0 con 0x7fcbb41032f0 2024-03-24T02:58:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.160+0000 7fcbb259c640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcb9c0a2880 0x7fcb9c0a4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcba40096c0 tx=0x7fcba402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:54.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.164+0000 7fcba37fe640 1 -- 172.21.15.17:0/1265360696 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcbac09c0f0 con 0x7fcbb41032f0 2024-03-24T02:58:54.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:54 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4163792526' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-24T02:58:54.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:54 smithi188 ceph-mon[36206]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 342 B/s wr, 0 op/s 2024-03-24T02:58:54.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:54 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4163792526' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-24T02:58:54.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:54 smithi019 ceph-mon[35944]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 342 B/s wr, 0 op/s 2024-03-24T02:58:54.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:54.387+0000 7fcbb8cea640 1 -- 172.21.15.17:0/1265360696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fcbb4062340 con 0x7fcbb41032f0 2024-03-24T02:58:54.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:54 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4163792526' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-24T02:58:54.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:54 smithi017 ceph-mon[31926]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 342 B/s wr, 0 op/s 2024-03-24T02:58:55.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.034+0000 7fcba37fe640 1 -- 172.21.15.17:0/1265360696 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fcbac09b9f0 con 0x7fcbb41032f0 2024-03-24T02:58:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.036+0000 7fcbb8cea640 1 -- 172.21.15.17:0/1265360696 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcb9c0a2880 msgr2=0x7fcb9c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.036+0000 7fcbb8cea640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcb9c0a2880 0x7fcb9c0a4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcba40096c0 tx=0x7fcba402df90 comp rx=0 tx=0).stop 2024-03-24T02:58:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.036+0000 7fcbb8cea640 1 -- 172.21.15.17:0/1265360696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb41032f0 msgr2=0x7fcbb4100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.036+0000 7fcbb8cea640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb41032f0 0x7fcbb4100e20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00eb10 tx=0x7fcbac00c2d0 comp rx=0 tx=0).stop 2024-03-24T02:58:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.036+0000 7fcbb8cea640 1 -- 172.21.15.17:0/1265360696 shutdown_connections 2024-03-24T02:58:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.037+0000 7fcbb8cea640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcb9c0a2880 0x7fcb9c0a4d40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.037+0000 7fcbb8cea640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb41032f0 0x7fcbb4100e20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.037+0000 7fcbb8cea640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbb4102930 0x7fcbb41008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.037+0000 7fcbb8cea640 1 --2- 172.21.15.17:0/1265360696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40ff960 0x7fcbb41022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.037+0000 7fcbb8cea640 1 -- 172.21.15.17:0/1265360696 >> 172.21.15.17:0/1265360696 conn(0x7fcbb40fb2b0 msgr2=0x7fcbb4105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.037+0000 7fcbb8cea640 1 -- 172.21.15.17:0/1265360696 shutdown_connections 2024-03-24T02:58:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.037+0000 7fcbb8cea640 1 -- 172.21.15.17:0/1265360696 wait complete. 2024-03-24T02:58:55.053 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-24T02:58:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.166+0000 7f9e06160640 1 -- 172.21.15.17:0/501777883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e001017b0 msgr2=0x7f9e00101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.166+0000 7f9e06160640 1 --2- 172.21.15.17:0/501777883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e001017b0 0x7f9e00101c30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009900 tx=0x7f9df402bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.166+0000 7f9e06160640 1 -- 172.21.15.17:0/501777883 shutdown_connections 2024-03-24T02:58:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.166+0000 7f9e06160640 1 --2- 172.21.15.17:0/501777883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e00102170 0x7f9e00108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.166+0000 7f9e06160640 1 --2- 172.21.15.17:0/501777883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e001017b0 0x7f9e00101c30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.166+0000 7f9e06160640 1 --2- 172.21.15.17:0/501777883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e001005c0 0x7f9e001009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.166+0000 7f9e06160640 1 -- 172.21.15.17:0/501777883 >> 172.21.15.17:0/501777883 conn(0x7f9e000fbb70 msgr2=0x7f9e000fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.166+0000 7f9e06160640 1 -- 172.21.15.17:0/501777883 shutdown_connections 2024-03-24T02:58:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.166+0000 7f9e06160640 1 -- 172.21.15.17:0/501777883 wait complete. 2024-03-24T02:58:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.167+0000 7f9e06160640 1 Processor -- start 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.167+0000 7f9e06160640 1 -- start start 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.167+0000 7f9e06160640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e001005c0 0x7f9e00196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.167+0000 7f9e06160640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e001017b0 0x7f9e00196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.167+0000 7f9e06160640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e00102170 0x7f9e0019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.167+0000 7f9e06160640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0010c240 con 0x7f9e001017b0 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.167+0000 7f9e06160640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0010c0c0 con 0x7f9e00102170 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.167+0000 7f9e06160640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0010c3c0 con 0x7f9e001005c0 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.167+0000 7f9dfeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e001017b0 0x7f9e00196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.167+0000 7f9dfeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e001017b0 0x7f9e00196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53228/0 (socket says 172.21.15.17:53228) 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.167+0000 7f9dfeffd640 1 -- 172.21.15.17:0/3051412673 learned_addr learned my addr 172.21.15.17:0/3051412673 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.167+0000 7f9dff7fe640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e001005c0 0x7f9e00196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9dfffff640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e00102170 0x7f9e0019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9dfeffd640 1 -- 172.21.15.17:0/3051412673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e001005c0 msgr2=0x7f9e00196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9dfeffd640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e001005c0 0x7f9e00196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9dfeffd640 1 -- 172.21.15.17:0/3051412673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e00102170 msgr2=0x7f9e0019ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9dfeffd640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e00102170 0x7f9e0019ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9dfeffd640 1 -- 172.21.15.17:0/3051412673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0019e440 con 0x7f9e001017b0 2024-03-24T02:58:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9dfffff640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e00102170 0x7f9e0019ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T02:58:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9dff7fe640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e001005c0 0x7f9e00196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:58:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9dfeffd640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e001017b0 0x7f9e00196d60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009230 tx=0x7f9df4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9dfcff9640 1 -- 172.21.15.17:0/3051412673 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40044d0 con 0x7f9e001017b0 2024-03-24T02:58:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9dfcff9640 1 -- 172.21.15.17:0/3051412673 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9df40054e0 con 0x7f9e001017b0 2024-03-24T02:58:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9e06160640 1 -- 172.21.15.17:0/3051412673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e00069cd0 con 0x7f9e001017b0 2024-03-24T02:58:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9dfcff9640 1 -- 172.21.15.17:0/3051412673 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40444e0 con 0x7f9e001017b0 2024-03-24T02:58:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.168+0000 7f9e06160640 1 -- 172.21.15.17:0/3051412673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0006a1c0 con 0x7f9e001017b0 2024-03-24T02:58:55.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.169+0000 7f9e06160640 1 -- 172.21.15.17:0/3051412673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e000688e0 con 0x7f9e001017b0 2024-03-24T02:58:55.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.170+0000 7f9dfcff9640 1 -- 172.21.15.17:0/3051412673 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9df40035f0 con 0x7f9e001017b0 2024-03-24T02:58:55.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.171+0000 7f9dfcff9640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9de80a29a0 0x7f9de80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:55.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.171+0000 7f9dff7fe640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9de80a29a0 0x7f9de80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:55.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.171+0000 7f9dfcff9640 1 -- 172.21.15.17:0/3051412673 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9df40f0a90 con 0x7f9e001017b0 2024-03-24T02:58:55.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.171+0000 7f9dff7fe640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9de80a29a0 0x7f9de80a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9e00101610 tx=0x7f9df0012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:55.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.175+0000 7f9dfcff9640 1 -- 172.21.15.17:0/3051412673 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9df40bd8f0 con 0x7f9e001017b0 2024-03-24T02:58:55.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:55 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4163792526' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-24T02:58:55.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:55 smithi188 ceph-mon[36206]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:55.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:55 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1265360696' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-24T02:58:55.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:55 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4163792526' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-24T02:58:55.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:55 smithi019 ceph-mon[35944]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:55.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:55 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1265360696' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-24T02:58:55.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:55.399+0000 7f9e06160640 1 -- 172.21.15.17:0/3051412673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f9e00107350 con 0x7f9e001017b0 2024-03-24T02:58:55.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:55 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4163792526' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-24T02:58:55.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:55 smithi017 ceph-mon[31926]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:55.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:55 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1265360696' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-24T02:58:56.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.043+0000 7f9dfcff9640 1 -- 172.21.15.17:0/3051412673 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f9df40bd1f0 con 0x7f9e001017b0 2024-03-24T02:58:56.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.046+0000 7f9e06160640 1 -- 172.21.15.17:0/3051412673 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9de80a29a0 msgr2=0x7f9de80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:56.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.046+0000 7f9e06160640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9de80a29a0 0x7f9de80a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9e00101610 tx=0x7f9df0012040 comp rx=0 tx=0).stop 2024-03-24T02:58:56.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.046+0000 7f9e06160640 1 -- 172.21.15.17:0/3051412673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e001017b0 msgr2=0x7f9e00196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:56.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.046+0000 7f9e06160640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e001017b0 0x7f9e00196d60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009230 tx=0x7f9df4005d50 comp rx=0 tx=0).stop 2024-03-24T02:58:56.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.046+0000 7f9e06160640 1 -- 172.21.15.17:0/3051412673 shutdown_connections 2024-03-24T02:58:56.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.046+0000 7f9e06160640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9de80a29a0 0x7f9de80a4e60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:56.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.046+0000 7f9e06160640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e00102170 0x7f9e0019ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:56.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.046+0000 7f9e06160640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e001017b0 0x7f9e00196d60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:56.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.046+0000 7f9e06160640 1 --2- 172.21.15.17:0/3051412673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e001005c0 0x7f9e00196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:56.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.046+0000 7f9e06160640 1 -- 172.21.15.17:0/3051412673 >> 172.21.15.17:0/3051412673 conn(0x7f9e000fbb70 msgr2=0x7f9e00104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:56.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.046+0000 7f9e06160640 1 -- 172.21.15.17:0/3051412673 shutdown_connections 2024-03-24T02:58:56.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.046+0000 7f9e06160640 1 -- 172.21.15.17:0/3051412673 wait complete. 2024-03-24T02:58:56.062 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-24T02:58:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.174+0000 7ff909b52640 1 -- 172.21.15.17:0/3061191522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9041028a0 msgr2=0x7ff904102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.174+0000 7ff909b52640 1 --2- 172.21.15.17:0/3061191522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9041028a0 0x7ff904102d20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009900 tx=0x7ff8f802bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.174+0000 7ff909b52640 1 -- 172.21.15.17:0/3061191522 shutdown_connections 2024-03-24T02:58:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.174+0000 7ff909b52640 1 --2- 172.21.15.17:0/3061191522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff904103260 0x7ff904109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.174+0000 7ff909b52640 1 --2- 172.21.15.17:0/3061191522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9041028a0 0x7ff904102d20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.174+0000 7ff909b52640 1 --2- 172.21.15.17:0/3061191522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9041016b0 0x7ff904101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.174+0000 7ff909b52640 1 -- 172.21.15.17:0/3061191522 >> 172.21.15.17:0/3061191522 conn(0x7ff9040fcc40 msgr2=0x7ff9040ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.174+0000 7ff909b52640 1 -- 172.21.15.17:0/3061191522 shutdown_connections 2024-03-24T02:58:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.174+0000 7ff909b52640 1 -- 172.21.15.17:0/3061191522 wait complete. 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.175+0000 7ff909b52640 1 Processor -- start 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.175+0000 7ff909b52640 1 -- start start 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.175+0000 7ff909b52640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9041016b0 0x7ff9041979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.175+0000 7ff909b52640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9041028a0 0x7ff904197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.175+0000 7ff909b52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff904103260 0x7ff90419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.175+0000 7ff909b52640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90410d380 con 0x7ff9041016b0 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.175+0000 7ff909b52640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90410d200 con 0x7ff904103260 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.175+0000 7ff909b52640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90410d500 con 0x7ff9041028a0 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.175+0000 7ff9037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9041016b0 0x7ff9041979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.175+0000 7ff9037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9041016b0 0x7ff9041979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53258/0 (socket says 172.21.15.17:53258) 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.175+0000 7ff9037fe640 1 -- 172.21.15.17:0/3801697831 learned_addr learned my addr 172.21.15.17:0/3801697831 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff902ffd640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9041028a0 0x7ff904197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff903fff640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff904103260 0x7ff90419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff9037fe640 1 -- 172.21.15.17:0/3801697831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9041028a0 msgr2=0x7ff904197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff9037fe640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9041028a0 0x7ff904197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff9037fe640 1 -- 172.21.15.17:0/3801697831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff904103260 msgr2=0x7ff90419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff9037fe640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff904103260 0x7ff90419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff9037fe640 1 -- 172.21.15.17:0/3801697831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90419f610 con 0x7ff9041016b0 2024-03-24T02:58:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff9037fe640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9041016b0 0x7ff9041979f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400e9c0 tx=0x7ff8f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff900ff9640 1 -- 172.21.15.17:0/3801697831 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400cd80 con 0x7ff9041016b0 2024-03-24T02:58:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff909b52640 1 -- 172.21.15.17:0/3801697831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90419f900 con 0x7ff9041016b0 2024-03-24T02:58:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff900ff9640 1 -- 172.21.15.17:0/3801697831 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8f40035f0 con 0x7ff9041016b0 2024-03-24T02:58:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff900ff9640 1 -- 172.21.15.17:0/3801697831 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4010690 con 0x7ff9041016b0 2024-03-24T02:58:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.176+0000 7ff909b52640 1 -- 172.21.15.17:0/3801697831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90419fe20 con 0x7ff9041016b0 2024-03-24T02:58:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.177+0000 7ff909b52640 1 -- 172.21.15.17:0/3801697831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9040688e0 con 0x7ff9041016b0 2024-03-24T02:58:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.178+0000 7ff900ff9640 1 -- 172.21.15.17:0/3801697831 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff8f40037a0 con 0x7ff9041016b0 2024-03-24T02:58:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.179+0000 7ff900ff9640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff8ec0a2800 0x7ff8ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.179+0000 7ff902ffd640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff8ec0a2800 0x7ff8ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.179+0000 7ff900ff9640 1 -- 172.21.15.17:0/3801697831 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff8f40cf050 con 0x7ff9041016b0 2024-03-24T02:58:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.179+0000 7ff902ffd640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff8ec0a2800 0x7ff8ec0a4cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff8f802c240 tx=0x7ff8f802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.184+0000 7ff900ff9640 1 -- 172.21.15.17:0/3801697831 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff8f409beb0 con 0x7ff9041016b0 2024-03-24T02:58:56.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:56 smithi188 ceph-mon[36206]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-24T02:58:56.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:56 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1265360696' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-24T02:58:56.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:56 smithi188 ceph-mon[36206]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-24T02:58:56.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:56 smithi188 ceph-mon[36206]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:56.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:56 smithi188 ceph-mon[36206]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-24T02:58:56.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:56 smithi188 ceph-mon[36206]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-24T02:58:56.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:56 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3051412673' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-24T02:58:56.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:56 smithi188 ceph-mon[36206]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-03-24T02:58:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:56 smithi019 ceph-mon[35944]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-24T02:58:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:56 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1265360696' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-24T02:58:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:56 smithi019 ceph-mon[35944]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-24T02:58:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:56 smithi019 ceph-mon[35944]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:56 smithi019 ceph-mon[35944]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-24T02:58:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:56 smithi019 ceph-mon[35944]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-24T02:58:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:56 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3051412673' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-24T02:58:56.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:56 smithi019 ceph-mon[35944]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-03-24T02:58:56.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:56.407+0000 7ff909b52640 1 -- 172.21.15.17:0/3801697831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7ff904108a50 con 0x7ff9041016b0 2024-03-24T02:58:56.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:56 smithi017 ceph-mon[31926]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-24T02:58:56.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:56 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1265360696' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-24T02:58:56.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:56 smithi017 ceph-mon[31926]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-24T02:58:56.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:56 smithi017 ceph-mon[31926]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-24T02:58:56.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:56 smithi017 ceph-mon[31926]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-24T02:58:56.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:56 smithi017 ceph-mon[31926]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-24T02:58:56.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:56 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3051412673' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-24T02:58:56.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:56 smithi017 ceph-mon[31926]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-03-24T02:58:57.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.050+0000 7ff900ff9640 1 -- 172.21.15.17:0/3801697831 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff8f409b7b0 con 0x7ff9041016b0 2024-03-24T02:58:57.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.053+0000 7ff909b52640 1 -- 172.21.15.17:0/3801697831 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff8ec0a2800 msgr2=0x7ff8ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.053+0000 7ff909b52640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff8ec0a2800 0x7ff8ec0a4cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff8f802c240 tx=0x7ff8f802df90 comp rx=0 tx=0).stop 2024-03-24T02:58:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.053+0000 7ff909b52640 1 -- 172.21.15.17:0/3801697831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9041016b0 msgr2=0x7ff9041979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.053+0000 7ff909b52640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9041016b0 0x7ff9041979f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400e9c0 tx=0x7ff8f400ee80 comp rx=0 tx=0).stop 2024-03-24T02:58:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.053+0000 7ff909b52640 1 -- 172.21.15.17:0/3801697831 shutdown_connections 2024-03-24T02:58:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.053+0000 7ff909b52640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff8ec0a2800 0x7ff8ec0a4cc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.053+0000 7ff909b52640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff904103260 0x7ff90419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.053+0000 7ff909b52640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9041028a0 0x7ff904197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.053+0000 7ff909b52640 1 --2- 172.21.15.17:0/3801697831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9041016b0 0x7ff9041979f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.053+0000 7ff909b52640 1 -- 172.21.15.17:0/3801697831 >> 172.21.15.17:0/3801697831 conn(0x7ff9040fcc40 msgr2=0x7ff904105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.053+0000 7ff909b52640 1 -- 172.21.15.17:0/3801697831 shutdown_connections 2024-03-24T02:58:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.053+0000 7ff909b52640 1 -- 172.21.15.17:0/3801697831 wait complete. 2024-03-24T02:58:57.069 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-03-24T02:58:57.070 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-24T02:58:57.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.182+0000 7fb054ee8640 1 -- 172.21.15.17:0/3546851199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb050102170 msgr2=0x7fb050108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:57.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.182+0000 7fb054ee8640 1 --2- 172.21.15.17:0/3546851199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb050102170 0x7fb050108a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb048009ca0 tx=0x7fb04802c0f0 comp rx=0 tx=0).stop 2024-03-24T02:58:57.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.182+0000 7fb054ee8640 1 -- 172.21.15.17:0/3546851199 shutdown_connections 2024-03-24T02:58:57.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.182+0000 7fb054ee8640 1 --2- 172.21.15.17:0/3546851199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb050102170 0x7fb050108a00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:57.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.182+0000 7fb054ee8640 1 --2- 172.21.15.17:0/3546851199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0501017b0 0x7fb050101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:57.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.182+0000 7fb054ee8640 1 --2- 172.21.15.17:0/3546851199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0501005c0 0x7fb0501009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:57.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.182+0000 7fb054ee8640 1 -- 172.21.15.17:0/3546851199 >> 172.21.15.17:0/3546851199 conn(0x7fb0500fbb50 msgr2=0x7fb0500fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:57.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.182+0000 7fb054ee8640 1 -- 172.21.15.17:0/3546851199 shutdown_connections 2024-03-24T02:58:57.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.182+0000 7fb054ee8640 1 -- 172.21.15.17:0/3546851199 wait complete. 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.183+0000 7fb054ee8640 1 Processor -- start 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.183+0000 7fb054ee8640 1 -- start start 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.183+0000 7fb054ee8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0501005c0 0x7fb050197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.183+0000 7fb054ee8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0501017b0 0x7fb050197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.183+0000 7fb054ee8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb050102170 0x7fb05019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.183+0000 7fb054ee8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05010c240 con 0x7fb0501017b0 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.183+0000 7fb054ee8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05010c0c0 con 0x7fb050102170 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.183+0000 7fb054ee8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05010c3c0 con 0x7fb0501005c0 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.183+0000 7fb047fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0501017b0 0x7fb050197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.183+0000 7fb047fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0501017b0 0x7fb050197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53284/0 (socket says 172.21.15.17:53284) 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.183+0000 7fb04e59c640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0501005c0 0x7fb050197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.183+0000 7fb047fff640 1 -- 172.21.15.17:0/1426262188 learned_addr learned my addr 172.21.15.17:0/1426262188 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.184+0000 7fb04ed9d640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb050102170 0x7fb05019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.184+0000 7fb047fff640 1 -- 172.21.15.17:0/1426262188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0501005c0 msgr2=0x7fb050197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.184+0000 7fb047fff640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0501005c0 0x7fb050197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.184+0000 7fb047fff640 1 -- 172.21.15.17:0/1426262188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb050102170 msgr2=0x7fb05019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.184+0000 7fb047fff640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb050102170 0x7fb05019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.184+0000 7fb047fff640 1 -- 172.21.15.17:0/1426262188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05019f630 con 0x7fb0501017b0 2024-03-24T02:58:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.184+0000 7fb047fff640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0501017b0 0x7fb050197fa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb03800e9f0 tx=0x7fb03800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.184+0000 7fb0477fe640 1 -- 172.21.15.17:0/1426262188 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03800cd80 con 0x7fb0501017b0 2024-03-24T02:58:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.184+0000 7fb054ee8640 1 -- 172.21.15.17:0/1426262188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05019f920 con 0x7fb0501017b0 2024-03-24T02:58:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.184+0000 7fb0477fe640 1 -- 172.21.15.17:0/1426262188 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb038005500 con 0x7fb0501017b0 2024-03-24T02:58:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.184+0000 7fb0477fe640 1 -- 172.21.15.17:0/1426262188 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038010690 con 0x7fb0501017b0 2024-03-24T02:58:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.184+0000 7fb054ee8640 1 -- 172.21.15.17:0/1426262188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05019fe70 con 0x7fb0501017b0 2024-03-24T02:58:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.185+0000 7fb054ee8640 1 -- 172.21.15.17:0/1426262188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0501a0100 con 0x7fb0501017b0 2024-03-24T02:58:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.186+0000 7fb0477fe640 1 -- 172.21.15.17:0/1426262188 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb0380046c0 con 0x7fb0501017b0 2024-03-24T02:58:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.187+0000 7fb0477fe640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb03c0a2800 0x7fb03c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.187+0000 7fb04e59c640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb03c0a2800 0x7fb03c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.187+0000 7fb0477fe640 1 -- 172.21.15.17:0/1426262188 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb0380cf0e0 con 0x7fb0501017b0 2024-03-24T02:58:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.187+0000 7fb04e59c640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb03c0a2800 0x7fb03c0a4cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb050101610 tx=0x7fb040005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.192+0000 7fb0477fe640 1 -- 172.21.15.17:0/1426262188 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb03809bee0 con 0x7fb0501017b0 2024-03-24T02:58:57.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:57 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3051412673' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-24T02:58:57.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:57 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] up:active 2024-03-24T02:58:57.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:57 smithi188 ceph-mon[36206]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-24T02:58:57.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:57 smithi188 ceph-mon[36206]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-24T02:58:57.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:57 smithi188 ceph-mon[36206]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-24T02:58:57.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:57 smithi188 ceph-mon[36206]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-24T02:58:57.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:57 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3801697831' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-24T02:58:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:57 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3051412673' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-24T02:58:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:57 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] up:active 2024-03-24T02:58:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:57 smithi019 ceph-mon[35944]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-24T02:58:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:57 smithi019 ceph-mon[35944]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-24T02:58:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:57 smithi019 ceph-mon[35944]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-24T02:58:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:57 smithi019 ceph-mon[35944]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-24T02:58:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:57 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3801697831' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-24T02:58:57.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:57.400+0000 7fb054ee8640 1 -- 172.21.15.17:0/1426262188 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0500fdc90 con 0x7fb03c0a2800 2024-03-24T02:58:57.405 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:57 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3051412673' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-24T02:58:57.405 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:57 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] up:active 2024-03-24T02:58:57.406 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:57 smithi017 ceph-mon[31926]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-24T02:58:57.406 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:57 smithi017 ceph-mon[31926]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-24T02:58:57.406 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:57 smithi017 ceph-mon[31926]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-24T02:58:57.406 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:57 smithi017 ceph-mon[31926]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-24T02:58:57.406 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:57 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3801697831' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-24T02:58:57.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:58:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:58:57.405+0000 7f5ebb9d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T02:58:57.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:58:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:58:57.405+0000 7f5ebb9d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T02:58:57.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:58:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:58:57.405+0000 7f5ebb9d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T02:58:57.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:58:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:58:57.405+0000 7f5ebb9d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T02:58:57.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 02:58:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T02:58:57.405+0000 7f5ebb9d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T02:58:58.047 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:58:57 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i[67156]: 2024-03-24T02:58:57.749+0000 7f913f57f700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-24T02:58:58.047 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:58:57 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i[67156]: 2024-03-24T02:58:57.749+0000 7f913f57f700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-24T02:58:58.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.090+0000 7fb0477fe640 1 -- 172.21.15.17:0/1426262188 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb0500fdc90 con 0x7fb03c0a2800 2024-03-24T02:58:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.092+0000 7fb054ee8640 1 -- 172.21.15.17:0/1426262188 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb03c0a2800 msgr2=0x7fb03c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.092+0000 7fb054ee8640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb03c0a2800 0x7fb03c0a4cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb050101610 tx=0x7fb040005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.092+0000 7fb054ee8640 1 -- 172.21.15.17:0/1426262188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0501017b0 msgr2=0x7fb050197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.092+0000 7fb054ee8640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0501017b0 0x7fb050197fa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb03800e9f0 tx=0x7fb03800edf0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.092+0000 7fb054ee8640 1 -- 172.21.15.17:0/1426262188 shutdown_connections 2024-03-24T02:58:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.092+0000 7fb054ee8640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb03c0a2800 0x7fb03c0a4cc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.092+0000 7fb054ee8640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb050102170 0x7fb05019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.092+0000 7fb054ee8640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0501017b0 0x7fb050197fa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.092+0000 7fb054ee8640 1 --2- 172.21.15.17:0/1426262188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0501005c0 0x7fb050197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.092+0000 7fb054ee8640 1 -- 172.21.15.17:0/1426262188 >> 172.21.15.17:0/1426262188 conn(0x7fb0500fbb50 msgr2=0x7fb0500fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.092+0000 7fb054ee8640 1 -- 172.21.15.17:0/1426262188 shutdown_connections 2024-03-24T02:58:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.092+0000 7fb054ee8640 1 -- 172.21.15.17:0/1426262188 wait complete. 2024-03-24T02:58:58.104 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-24T02:58:58.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.214+0000 7f6447a97640 1 -- 172.21.15.17:0/2020971510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64401017a0 msgr2=0x7f6440101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.214+0000 7f6447a97640 1 --2- 172.21.15.17:0/2020971510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64401017a0 0x7f6440101c20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6434009900 tx=0x7f643402bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:58.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 -- 172.21.15.17:0/2020971510 shutdown_connections 2024-03-24T02:58:58.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 --2- 172.21.15.17:0/2020971510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6440102160 0x7f64401089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 --2- 172.21.15.17:0/2020971510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64401017a0 0x7f6440101c20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 --2- 172.21.15.17:0/2020971510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64401005b0 0x7f64401009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 -- 172.21.15.17:0/2020971510 >> 172.21.15.17:0/2020971510 conn(0x7f64400fbb40 msgr2=0x7f64400fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:58.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 -- 172.21.15.17:0/2020971510 shutdown_connections 2024-03-24T02:58:58.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 -- 172.21.15.17:0/2020971510 wait complete. 2024-03-24T02:58:58.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 Processor -- start 2024-03-24T02:58:58.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 -- start start 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64401005b0 0x7f644006a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64401017a0 0x7f644019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6440102160 0x7f644019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644010c210 con 0x7f64401005b0 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644010c090 con 0x7f64401017a0 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6447a97640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644010c390 con 0x7f6440102160 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6445833640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64401005b0 0x7f644006a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6445833640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64401005b0 0x7f644006a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53314/0 (socket says 172.21.15.17:53314) 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6445833640 1 -- 172.21.15.17:0/1163999407 learned_addr learned my addr 172.21.15.17:0/1163999407 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6446034640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6440102160 0x7f644019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.215+0000 7f6445032640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64401017a0 0x7f644019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.216+0000 7f6445833640 1 -- 172.21.15.17:0/1163999407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6440102160 msgr2=0x7f644019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.216+0000 7f6445833640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6440102160 0x7f644019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.216+0000 7f6445833640 1 -- 172.21.15.17:0/1163999407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64401017a0 msgr2=0x7f644019c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.216+0000 7f6445833640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64401017a0 0x7f644019c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.216+0000 7f6445833640 1 -- 172.21.15.17:0/1163999407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644019f640 con 0x7f64401005b0 2024-03-24T02:58:58.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.216+0000 7f6445833640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64401005b0 0x7f644006a800 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f643800e9c0 tx=0x7f643800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:58.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.216+0000 7f643effd640 1 -- 172.21.15.17:0/1163999407 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643800ccc0 con 0x7f64401005b0 2024-03-24T02:58:58.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.216+0000 7f643effd640 1 -- 172.21.15.17:0/1163999407 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64380035f0 con 0x7f64401005b0 2024-03-24T02:58:58.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.216+0000 7f6447a97640 1 -- 172.21.15.17:0/1163999407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644019f930 con 0x7f64401005b0 2024-03-24T02:58:58.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.216+0000 7f643effd640 1 -- 172.21.15.17:0/1163999407 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643800ce60 con 0x7f64401005b0 2024-03-24T02:58:58.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.216+0000 7f6447a97640 1 -- 172.21.15.17:0/1163999407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644019fe50 con 0x7f64401005b0 2024-03-24T02:58:58.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.217+0000 7f6447a97640 1 -- 172.21.15.17:0/1163999407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6440107950 con 0x7f64401005b0 2024-03-24T02:58:58.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.218+0000 7f643effd640 1 -- 172.21.15.17:0/1163999407 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f64380037a0 con 0x7f64401005b0 2024-03-24T02:58:58.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.219+0000 7f643effd640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f64280a2800 0x7f64280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:58.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.219+0000 7f6445032640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f64280a2800 0x7f64280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:58.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.219+0000 7f643effd640 1 -- 172.21.15.17:0/1163999407 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f64380cf120 con 0x7f64401005b0 2024-03-24T02:58:58.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.219+0000 7f6445032640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f64280a2800 0x7f64280a4cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f643402c7b0 tx=0x7f6434005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:58.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.222+0000 7f643effd640 1 -- 172.21.15.17:0/1163999407 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f643809bf80 con 0x7f64401005b0 2024-03-24T02:58:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:58 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3801697831' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-24T02:58:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:58 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] up:active 2024-03-24T02:58:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:58 smithi019 ceph-mon[35944]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-24T02:58:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:58 smithi019 ceph-mon[35944]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-24T02:58:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:58 smithi019 ceph-mon[35944]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-03-24T02:58:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:58 smithi019 ceph-mon[35944]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-24T02:58:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:58 smithi019 ceph-mon[35944]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:58:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:58 smithi019 ceph-mon[35944]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 3 op/s 2024-03-24T02:58:58.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.438+0000 7f6447a97640 1 -- 172.21.15.17:0/1163999407 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f6440061ae0 con 0x7f64280a2800 2024-03-24T02:58:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:58 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3801697831' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-24T02:58:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:58 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] up:active 2024-03-24T02:58:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:58 smithi017 ceph-mon[31926]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-24T02:58:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:58 smithi017 ceph-mon[31926]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-24T02:58:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:58 smithi017 ceph-mon[31926]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-03-24T02:58:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:58 smithi017 ceph-mon[31926]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-24T02:58:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:58 smithi017 ceph-mon[31926]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:58:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:58 smithi017 ceph-mon[31926]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 3 op/s 2024-03-24T02:58:58.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.466+0000 7f643effd640 1 -- 172.21.15.17:0/1163999407 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6440061ae0 con 0x7f64280a2800 2024-03-24T02:58:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.468+0000 7f6447a97640 1 -- 172.21.15.17:0/1163999407 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f64280a2800 msgr2=0x7f64280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.468+0000 7f6447a97640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f64280a2800 0x7f64280a4cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f643402c7b0 tx=0x7f6434005fd0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.468+0000 7f6447a97640 1 -- 172.21.15.17:0/1163999407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64401005b0 msgr2=0x7f644006a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.468+0000 7f6447a97640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64401005b0 0x7f644006a800 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f643800e9c0 tx=0x7f643800edc0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.468+0000 7f6447a97640 1 -- 172.21.15.17:0/1163999407 shutdown_connections 2024-03-24T02:58:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.468+0000 7f6447a97640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f64280a2800 0x7f64280a4cc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.468+0000 7f6447a97640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6440102160 0x7f644019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.468+0000 7f6447a97640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64401017a0 0x7f644019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.468+0000 7f6447a97640 1 --2- 172.21.15.17:0/1163999407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64401005b0 0x7f644006a800 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.468+0000 7f6447a97640 1 -- 172.21.15.17:0/1163999407 >> 172.21.15.17:0/1163999407 conn(0x7f64400fbb40 msgr2=0x7f6440104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.468+0000 7f6447a97640 1 -- 172.21.15.17:0/1163999407 shutdown_connections 2024-03-24T02:58:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.468+0000 7f6447a97640 1 -- 172.21.15.17:0/1163999407 wait complete. 2024-03-24T02:58:58.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-24T02:58:58.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:58 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3801697831' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-24T02:58:58.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:58 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] up:active 2024-03-24T02:58:58.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:58 smithi188 ceph-mon[36206]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-24T02:58:58.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:58 smithi188 ceph-mon[36206]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-24T02:58:58.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:58 smithi188 ceph-mon[36206]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-03-24T02:58:58.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:58 smithi188 ceph-mon[36206]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-24T02:58:58.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:58 smithi188 ceph-mon[36206]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:58:58.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:58 smithi188 ceph-mon[36206]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 3 op/s 2024-03-24T02:58:58.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.586+0000 7f6236d53640 1 -- 172.21.15.17:0/3910364918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62301005a0 msgr2=0x7f62301009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.586+0000 7f6236d53640 1 --2- 172.21.15.17:0/3910364918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62301005a0 0x7f62301009c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f622c009900 tx=0x7f622c02bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:58.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.586+0000 7f6236d53640 1 -- 172.21.15.17:0/3910364918 shutdown_connections 2024-03-24T02:58:58.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.586+0000 7f6236d53640 1 --2- 172.21.15.17:0/3910364918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6230102150 0x7f62301089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.586+0000 7f6236d53640 1 --2- 172.21.15.17:0/3910364918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6230101790 0x7f6230101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.586+0000 7f6236d53640 1 --2- 172.21.15.17:0/3910364918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62301005a0 0x7f62301009c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.586+0000 7f6236d53640 1 -- 172.21.15.17:0/3910364918 >> 172.21.15.17:0/3910364918 conn(0x7f62300fbb30 msgr2=0x7f62300fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:58.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.586+0000 7f6236d53640 1 -- 172.21.15.17:0/3910364918 shutdown_connections 2024-03-24T02:58:58.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.586+0000 7f6236d53640 1 -- 172.21.15.17:0/3910364918 wait complete. 2024-03-24T02:58:58.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6236d53640 1 Processor -- start 2024-03-24T02:58:58.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6236d53640 1 -- start start 2024-03-24T02:58:58.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6236d53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62301005a0 0x7f6230196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:58.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6236d53640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6230101790 0x7f6230196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:58.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6236d53640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6230102150 0x7f623019deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6236d53640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623010c230 con 0x7f6230101790 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6236d53640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623010c0b0 con 0x7f62301005a0 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6236d53640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623010c3b0 con 0x7f6230102150 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6227fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6230101790 0x7f6230196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6227fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6230101790 0x7f6230196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53332/0 (socket says 172.21.15.17:53332) 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6234aef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62301005a0 0x7f6230196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6227fff640 1 -- 172.21.15.17:0/1595845352 learned_addr learned my addr 172.21.15.17:0/1595845352 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f62352f0640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6230102150 0x7f623019deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6227fff640 1 -- 172.21.15.17:0/1595845352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6230102150 msgr2=0x7f623019deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6227fff640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6230102150 0x7f623019deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6227fff640 1 -- 172.21.15.17:0/1595845352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62301005a0 msgr2=0x7f6230196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6227fff640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62301005a0 0x7f6230196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6227fff640 1 -- 172.21.15.17:0/1595845352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623019e520 con 0x7f6230101790 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.587+0000 7f6234aef640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62301005a0 0x7f6230196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.588+0000 7f6227fff640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6230101790 0x7f6230196e40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6220009f50 tx=0x7f622000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:58.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.588+0000 7f6225ffb640 1 -- 172.21.15.17:0/1595845352 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622001e070 con 0x7f6230101790 2024-03-24T02:58:58.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.588+0000 7f6225ffb640 1 -- 172.21.15.17:0/1595845352 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f622000cce0 con 0x7f6230101790 2024-03-24T02:58:58.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.588+0000 7f6236d53640 1 -- 172.21.15.17:0/1595845352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6230068860 con 0x7f6230101790 2024-03-24T02:58:58.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.588+0000 7f6225ffb640 1 -- 172.21.15.17:0/1595845352 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220019900 con 0x7f6230101790 2024-03-24T02:58:58.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.588+0000 7f6236d53640 1 -- 172.21.15.17:0/1595845352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6230068e00 con 0x7f6230101790 2024-03-24T02:58:58.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.589+0000 7f6236d53640 1 -- 172.21.15.17:0/1595845352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6230107450 con 0x7f6230101790 2024-03-24T02:58:58.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.590+0000 7f6225ffb640 1 -- 172.21.15.17:0/1595845352 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6220021070 con 0x7f6230101790 2024-03-24T02:58:58.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.591+0000 7f6225ffb640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f62180a2880 0x7f62180a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:58.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.591+0000 7f6234aef640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f62180a2880 0x7f62180a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:58.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.591+0000 7f6225ffb640 1 -- 172.21.15.17:0/1595845352 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f62200d0c30 con 0x7f6230101790 2024-03-24T02:58:58.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.591+0000 7f6234aef640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f62180a2880 0x7f62180a4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f622c000c00 tx=0x7f622c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:58.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.595+0000 7f6225ffb640 1 -- 172.21.15.17:0/1595845352 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f622009d3e0 con 0x7f6230101790 2024-03-24T02:58:58.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.808+0000 7f6236d53640 1 -- 172.21.15.17:0/1595845352 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6230069150 con 0x7f62180a2880 2024-03-24T02:58:58.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.817+0000 7f6225ffb640 1 -- 172.21.15.17:0/1595845352 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f6230069150 con 0x7f62180a2880 2024-03-24T02:58:58.816 INFO:teuthology.orchestra.run.smithi017.stdout:/volumes/qa/sv_0/67238cd0-0016-47e4-a49c-5f911cca9d2c 2024-03-24T02:58:58.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.819+0000 7f6236d53640 1 -- 172.21.15.17:0/1595845352 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f62180a2880 msgr2=0x7f62180a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.819+0000 7f6236d53640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f62180a2880 0x7f62180a4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f622c000c00 tx=0x7f622c005fd0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.819+0000 7f6236d53640 1 -- 172.21.15.17:0/1595845352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6230101790 msgr2=0x7f6230196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.819+0000 7f6236d53640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6230101790 0x7f6230196e40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6220009f50 tx=0x7f622000ee50 comp rx=0 tx=0).stop 2024-03-24T02:58:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.819+0000 7f6236d53640 1 -- 172.21.15.17:0/1595845352 shutdown_connections 2024-03-24T02:58:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.819+0000 7f6236d53640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f62180a2880 0x7f62180a4d40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.819+0000 7f6236d53640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6230102150 0x7f623019deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.819+0000 7f6236d53640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6230101790 0x7f6230196e40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.819+0000 7f6236d53640 1 --2- 172.21.15.17:0/1595845352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62301005a0 0x7f6230196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.819+0000 7f6236d53640 1 -- 172.21.15.17:0/1595845352 >> 172.21.15.17:0/1595845352 conn(0x7f62300fbb30 msgr2=0x7f6230104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.819+0000 7f6236d53640 1 -- 172.21.15.17:0/1595845352 shutdown_connections 2024-03-24T02:58:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.819+0000 7f6236d53640 1 -- 172.21.15.17:0/1595845352 wait complete. 2024-03-24T02:58:58.829 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-24T02:58:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.937+0000 7fcc6ea36640 1 -- 172.21.15.17:0/1062406274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc681021c0 msgr2=0x7fcc681045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.937+0000 7fcc6ea36640 1 --2- 172.21.15.17:0/1062406274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc681021c0 0x7fcc681045b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009900 tx=0x7fcc5c02bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.937+0000 7fcc6ea36640 1 -- 172.21.15.17:0/1062406274 shutdown_connections 2024-03-24T02:58:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.937+0000 7fcc6ea36640 1 --2- 172.21.15.17:0/1062406274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc68107420 0x7fcc68109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.937+0000 7fcc6ea36640 1 --2- 172.21.15.17:0/1062406274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc68104af0 0x7fcc68106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.937+0000 7fcc6ea36640 1 --2- 172.21.15.17:0/1062406274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc681021c0 0x7fcc681045b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.937+0000 7fcc6ea36640 1 -- 172.21.15.17:0/1062406274 >> 172.21.15.17:0/1062406274 conn(0x7fcc680fbb30 msgr2=0x7fcc680fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.937+0000 7fcc6ea36640 1 -- 172.21.15.17:0/1062406274 shutdown_connections 2024-03-24T02:58:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.937+0000 7fcc6ea36640 1 -- 172.21.15.17:0/1062406274 wait complete. 2024-03-24T02:58:58.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc6ea36640 1 Processor -- start 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc6ea36640 1 -- start start 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc6ea36640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc681021c0 0x7fcc68101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc6ea36640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc68104af0 0x7fcc68100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc6ea36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc68107420 0x7fcc681006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc6ea36640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6810d340 con 0x7fcc681021c0 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc6ea36640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6810d1c0 con 0x7fcc68107420 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc6ea36640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6810d4c0 con 0x7fcc68104af0 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc67fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc681021c0 0x7fcc68101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc67fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc681021c0 0x7fcc68101b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53354/0 (socket says 172.21.15.17:53354) 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc67fff640 1 -- 172.21.15.17:0/2028298129 learned_addr learned my addr 172.21.15.17:0/2028298129 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc677fe640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc68104af0 0x7fcc68100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.938+0000 7fcc6cfd3640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc68107420 0x7fcc681006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.939+0000 7fcc67fff640 1 -- 172.21.15.17:0/2028298129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc68104af0 msgr2=0x7fcc68100170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.939+0000 7fcc67fff640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc68104af0 0x7fcc68100170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.939+0000 7fcc67fff640 1 -- 172.21.15.17:0/2028298129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc68107420 msgr2=0x7fcc681006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.939+0000 7fcc67fff640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc68107420 0x7fcc681006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.939+0000 7fcc67fff640 1 -- 172.21.15.17:0/2028298129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68100f40 con 0x7fcc681021c0 2024-03-24T02:58:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.939+0000 7fcc677fe640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc68104af0 0x7fcc68100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:58:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.939+0000 7fcc67fff640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc681021c0 0x7fcc68101b30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009230 tx=0x7fcc5c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.939+0000 7fcc657fa640 1 -- 172.21.15.17:0/2028298129 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c041070 con 0x7fcc681021c0 2024-03-24T02:58:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.939+0000 7fcc6ea36640 1 -- 172.21.15.17:0/2028298129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc681a1b00 con 0x7fcc681021c0 2024-03-24T02:58:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.939+0000 7fcc657fa640 1 -- 172.21.15.17:0/2028298129 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc5c004450 con 0x7fcc681021c0 2024-03-24T02:58:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.939+0000 7fcc657fa640 1 -- 172.21.15.17:0/2028298129 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c003e20 con 0x7fcc681021c0 2024-03-24T02:58:58.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.939+0000 7fcc6ea36640 1 -- 172.21.15.17:0/2028298129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc681a2050 con 0x7fcc681021c0 2024-03-24T02:58:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.940+0000 7fcc6ea36640 1 -- 172.21.15.17:0/2028298129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc680fd620 con 0x7fcc681021c0 2024-03-24T02:58:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.941+0000 7fcc657fa640 1 -- 172.21.15.17:0/2028298129 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fcc5c044070 con 0x7fcc681021c0 2024-03-24T02:58:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.942+0000 7fcc657fa640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcc500a2800 0x7fcc500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.942+0000 7fcc657fa640 1 -- 172.21.15.17:0/2028298129 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fcc5c0f0840 con 0x7fcc681021c0 2024-03-24T02:58:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.942+0000 7fcc677fe640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcc500a2800 0x7fcc500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:58.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.942+0000 7fcc677fe640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcc500a2800 0x7fcc500a4cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc681018a0 tx=0x7fcc58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:58.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:58.946+0000 7fcc657fa640 1 -- 172.21.15.17:0/2028298129 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc5c0bd050 con 0x7fcc681021c0 2024-03-24T02:58:59.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.089+0000 7fcc657fa640 1 -- 172.21.15.17:0/2028298129 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fcc5c0035f0 con 0x7fcc681021c0 2024-03-24T02:58:59.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.166+0000 7fcc6ea36640 1 -- 172.21.15.17:0/2028298129 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc68061b70 con 0x7fcc500a2800 2024-03-24T02:58:59.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.194+0000 7fcc657fa640 1 -- 172.21.15.17:0/2028298129 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcc68061b70 con 0x7fcc500a2800 2024-03-24T02:58:59.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.196+0000 7fcc6ea36640 1 -- 172.21.15.17:0/2028298129 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcc500a2800 msgr2=0x7fcc500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:59.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.196+0000 7fcc6ea36640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcc500a2800 0x7fcc500a4cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc681018a0 tx=0x7fcc58005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:59.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.196+0000 7fcc6ea36640 1 -- 172.21.15.17:0/2028298129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc681021c0 msgr2=0x7fcc68101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:59.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.196+0000 7fcc6ea36640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc681021c0 0x7fcc68101b30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009230 tx=0x7fcc5c02c650 comp rx=0 tx=0).stop 2024-03-24T02:58:59.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.196+0000 7fcc6ea36640 1 -- 172.21.15.17:0/2028298129 shutdown_connections 2024-03-24T02:58:59.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.196+0000 7fcc6ea36640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fcc500a2800 0x7fcc500a4cc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.196+0000 7fcc6ea36640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc68107420 0x7fcc681006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.196+0000 7fcc6ea36640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc68104af0 0x7fcc68100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.196+0000 7fcc6ea36640 1 --2- 172.21.15.17:0/2028298129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc681021c0 0x7fcc68101b30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.196+0000 7fcc6ea36640 1 -- 172.21.15.17:0/2028298129 >> 172.21.15.17:0/2028298129 conn(0x7fcc680fbb30 msgr2=0x7fcc68102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:59.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.196+0000 7fcc6ea36640 1 -- 172.21.15.17:0/2028298129 shutdown_connections 2024-03-24T02:58:59.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.196+0000 7fcc6ea36640 1 -- 172.21.15.17:0/2028298129 wait complete. 2024-03-24T02:58:59.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:59 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] up:active 2024-03-24T02:58:59.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:59 smithi017 ceph-mon[31926]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-24T02:58:59.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:59 smithi017 ceph-mon[31926]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-24T02:58:59.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:59 smithi017 ceph-mon[31926]: Cluster is now healthy 2024-03-24T02:58:59.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:59 smithi017 ceph-mon[31926]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-03-24T02:58:59.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:59 smithi017 ceph-mon[31926]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-24T02:58:59.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:59 smithi017 ceph-mon[31926]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-03-24T02:58:59.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:59 smithi017 ceph-mon[31926]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:58:59.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:58:59 smithi017 ceph-mon[31926]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:58:59.207 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-24T02:58:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 -- 172.21.15.17:0/1006130980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5441028a0 msgr2=0x7fe544102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 --2- 172.21.15.17:0/1006130980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5441028a0 0x7fe544102d20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe538009900 tx=0x7fe53802bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 -- 172.21.15.17:0/1006130980 shutdown_connections 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 --2- 172.21.15.17:0/1006130980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe544103260 0x7fe544109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 --2- 172.21.15.17:0/1006130980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5441028a0 0x7fe544102d20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 --2- 172.21.15.17:0/1006130980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5441016b0 0x7fe544101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 -- 172.21.15.17:0/1006130980 >> 172.21.15.17:0/1006130980 conn(0x7fe5440fcc40 msgr2=0x7fe5440ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 -- 172.21.15.17:0/1006130980 shutdown_connections 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 -- 172.21.15.17:0/1006130980 wait complete. 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 Processor -- start 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 -- start start 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5441016b0 0x7fe5441979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5441028a0 0x7fe544197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe544103260 0x7fe54419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54410d310 con 0x7fe5441028a0 2024-03-24T02:58:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54410d190 con 0x7fe5441016b0 2024-03-24T02:58:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.315+0000 7fe54b73b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54410d490 con 0x7fe544103260 2024-03-24T02:58:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe548cd6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5441028a0 0x7fe544197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe5494d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5441016b0 0x7fe5441979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe548cd6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5441028a0 0x7fe544197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53374/0 (socket says 172.21.15.17:53374) 2024-03-24T02:58:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe548cd6640 1 -- 172.21.15.17:0/1149500591 learned_addr learned my addr 172.21.15.17:0/1149500591 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe549cd8640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe544103260 0x7fe54419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe548cd6640 1 -- 172.21.15.17:0/1149500591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe544103260 msgr2=0x7fe54419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe548cd6640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe544103260 0x7fe54419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe548cd6640 1 -- 172.21.15.17:0/1149500591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5441016b0 msgr2=0x7fe5441979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe548cd6640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5441016b0 0x7fe5441979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe548cd6640 1 -- 172.21.15.17:0/1149500591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54419f5d0 con 0x7fe5441028a0 2024-03-24T02:58:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe5494d7640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5441016b0 0x7fe5441979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:58:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe548cd6640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5441028a0 0x7fe544197ef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe538009230 tx=0x7fe53802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe5427fc640 1 -- 172.21.15.17:0/1149500591 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538041070 con 0x7fe5441028a0 2024-03-24T02:58:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe5427fc640 1 -- 172.21.15.17:0/1149500591 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe538004450 con 0x7fe5441028a0 2024-03-24T02:58:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe54b73b640 1 -- 172.21.15.17:0/1149500591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54419f860 con 0x7fe5441028a0 2024-03-24T02:58:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe5427fc640 1 -- 172.21.15.17:0/1149500591 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538003e20 con 0x7fe5441028a0 2024-03-24T02:58:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.316+0000 7fe54b73b640 1 -- 172.21.15.17:0/1149500591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54419fd50 con 0x7fe5441028a0 2024-03-24T02:58:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.317+0000 7fe54b73b640 1 -- 172.21.15.17:0/1149500591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe544108a50 con 0x7fe5441028a0 2024-03-24T02:58:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.318+0000 7fe5427fc640 1 -- 172.21.15.17:0/1149500591 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fe538044070 con 0x7fe5441028a0 2024-03-24T02:58:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.319+0000 7fe5427fc640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe52c0a2890 0x7fe52c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.319+0000 7fe5494d7640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe52c0a2890 0x7fe52c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.319+0000 7fe5427fc640 1 -- 172.21.15.17:0/1149500591 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe5380f08b0 con 0x7fe5441028a0 2024-03-24T02:58:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.319+0000 7fe5494d7640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe52c0a2890 0x7fe52c0a4d50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe544102700 tx=0x7fe53c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:59.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:59 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] up:active 2024-03-24T02:58:59.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:59 smithi019 ceph-mon[35944]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-24T02:58:59.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:59 smithi019 ceph-mon[35944]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-24T02:58:59.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:59 smithi019 ceph-mon[35944]: Cluster is now healthy 2024-03-24T02:58:59.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:59 smithi019 ceph-mon[35944]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-03-24T02:58:59.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:59 smithi019 ceph-mon[35944]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-24T02:58:59.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:59 smithi019 ceph-mon[35944]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-03-24T02:58:59.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:59 smithi019 ceph-mon[35944]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:58:59.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:58:59 smithi019 ceph-mon[35944]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:58:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.323+0000 7fe5427fc640 1 -- 172.21.15.17:0/1149500591 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe5380bd070 con 0x7fe5441028a0 2024-03-24T02:58:59.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.537+0000 7fe54b73b640 1 -- 172.21.15.17:0/1149500591 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe544061b70 con 0x7fe52c0a2890 2024-03-24T02:58:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.544+0000 7fe5427fc640 1 -- 172.21.15.17:0/1149500591 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fe544061b70 con 0x7fe52c0a2890 2024-03-24T02:58:59.543 INFO:teuthology.orchestra.run.smithi017.stdout:/volumes/qa/sv_1/04bb16c3-438e-4302-875a-85564d3f557c 2024-03-24T02:58:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.546+0000 7fe54b73b640 1 -- 172.21.15.17:0/1149500591 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe52c0a2890 msgr2=0x7fe52c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.546+0000 7fe54b73b640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe52c0a2890 0x7fe52c0a4d50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe544102700 tx=0x7fe53c005f90 comp rx=0 tx=0).stop 2024-03-24T02:58:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.546+0000 7fe54b73b640 1 -- 172.21.15.17:0/1149500591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5441028a0 msgr2=0x7fe544197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.546+0000 7fe54b73b640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5441028a0 0x7fe544197ef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe538009230 tx=0x7fe53802c650 comp rx=0 tx=0).stop 2024-03-24T02:58:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.546+0000 7fe54b73b640 1 -- 172.21.15.17:0/1149500591 shutdown_connections 2024-03-24T02:58:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.546+0000 7fe54b73b640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe52c0a2890 0x7fe52c0a4d50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.546+0000 7fe54b73b640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe544103260 0x7fe54419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.546+0000 7fe54b73b640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5441028a0 0x7fe544197ef0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.546+0000 7fe54b73b640 1 --2- 172.21.15.17:0/1149500591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5441016b0 0x7fe5441979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.546+0000 7fe54b73b640 1 -- 172.21.15.17:0/1149500591 >> 172.21.15.17:0/1149500591 conn(0x7fe5440fcc40 msgr2=0x7fe544105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.546+0000 7fe54b73b640 1 -- 172.21.15.17:0/1149500591 shutdown_connections 2024-03-24T02:58:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.546+0000 7fe54b73b640 1 -- 172.21.15.17:0/1149500591 wait complete. 2024-03-24T02:58:59.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:59 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] up:active 2024-03-24T02:58:59.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:59 smithi188 ceph-mon[36206]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-24T02:58:59.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:59 smithi188 ceph-mon[36206]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-24T02:58:59.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:59 smithi188 ceph-mon[36206]: Cluster is now healthy 2024-03-24T02:58:59.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:59 smithi188 ceph-mon[36206]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-03-24T02:58:59.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:59 smithi188 ceph-mon[36206]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-24T02:58:59.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:59 smithi188 ceph-mon[36206]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-03-24T02:58:59.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:59 smithi188 ceph-mon[36206]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:58:59.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:58:59 smithi188 ceph-mon[36206]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:58:59.556 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T02:58:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.659+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/611526687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b28104a70 msgr2=0x7f2b28106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.659+0000 7f2b2f8f1640 1 --2- 172.21.15.17:0/611526687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b28104a70 0x7f2b28106e60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009900 tx=0x7f2b1c02bc30 comp rx=0 tx=0).stop 2024-03-24T02:58:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.659+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/611526687 shutdown_connections 2024-03-24T02:58:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.659+0000 7f2b2f8f1640 1 --2- 172.21.15.17:0/611526687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b281073a0 0x7f2b281097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.659+0000 7f2b2f8f1640 1 --2- 172.21.15.17:0/611526687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b28104a70 0x7f2b28106e60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.659+0000 7f2b2f8f1640 1 --2- 172.21.15.17:0/611526687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b28102140 0x7f2b28104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.659+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/611526687 >> 172.21.15.17:0/611526687 conn(0x7f2b280fbb50 msgr2=0x7f2b280fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.659+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/611526687 shutdown_connections 2024-03-24T02:58:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.659+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/611526687 wait complete. 2024-03-24T02:58:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2f8f1640 1 Processor -- start 2024-03-24T02:58:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2f8f1640 1 -- start start 2024-03-24T02:58:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2f8f1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b28102140 0x7f2b28197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2f8f1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b28104a70 0x7f2b28197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2f8f1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b281073a0 0x7f2b2819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2f8f1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2810d3a0 con 0x7f2b28102140 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2f8f1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2810d220 con 0x7f2b28104a70 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2f8f1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2810d520 con 0x7f2b281073a0 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2d68d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b28102140 0x7f2b28197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2d68d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b28102140 0x7f2b28197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53394/0 (socket says 172.21.15.17:53394) 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2d68d640 1 -- 172.21.15.17:0/3448624485 learned_addr learned my addr 172.21.15.17:0/3448624485 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2de8e640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b281073a0 0x7f2b2819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.660+0000 7f2b2ce8c640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b28104a70 0x7f2b28197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.661+0000 7f2b2d68d640 1 -- 172.21.15.17:0/3448624485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b281073a0 msgr2=0x7f2b2819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.661+0000 7f2b2d68d640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b281073a0 0x7f2b2819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.661+0000 7f2b2d68d640 1 -- 172.21.15.17:0/3448624485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b28104a70 msgr2=0x7f2b28197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.661+0000 7f2b2d68d640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b28104a70 0x7f2b28197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.661+0000 7f2b2d68d640 1 -- 172.21.15.17:0/3448624485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2819f630 con 0x7f2b28102140 2024-03-24T02:58:59.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.661+0000 7f2b2de8e640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b281073a0 0x7f2b2819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:58:59.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.661+0000 7f2b2d68d640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b28102140 0x7f2b28197a10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000e9f0 tx=0x7f2b2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:59.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.661+0000 7f2b267fc640 1 -- 172.21.15.17:0/3448624485 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2000cd80 con 0x7f2b28102140 2024-03-24T02:58:59.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.661+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/3448624485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2819f920 con 0x7f2b28102140 2024-03-24T02:58:59.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.661+0000 7f2b267fc640 1 -- 172.21.15.17:0/3448624485 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b20005500 con 0x7f2b28102140 2024-03-24T02:58:59.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.661+0000 7f2b267fc640 1 -- 172.21.15.17:0/3448624485 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20010690 con 0x7f2b28102140 2024-03-24T02:58:59.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.661+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/3448624485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2819fdf0 con 0x7f2b28102140 2024-03-24T02:58:59.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.662+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/3448624485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b280fd640 con 0x7f2b28102140 2024-03-24T02:58:59.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.663+0000 7f2b267fc640 1 -- 172.21.15.17:0/3448624485 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f2b200046c0 con 0x7f2b28102140 2024-03-24T02:58:59.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.663+0000 7f2b267fc640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2b100a2890 0x7f2b100a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:58:59.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.664+0000 7f2b2ce8c640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2b100a2890 0x7f2b100a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:58:59.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.664+0000 7f2b267fc640 1 -- 172.21.15.17:0/3448624485 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2b200cf220 con 0x7f2b28102140 2024-03-24T02:58:59.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.664+0000 7f2b2ce8c640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2b100a2890 0x7f2b100a4d50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c02c170 tx=0x7f2b1c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:58:59.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.668+0000 7f2b267fc640 1 -- 172.21.15.17:0/3448624485 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2b2009bf20 con 0x7f2b28102140 2024-03-24T02:58:59.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.891+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/3448624485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b2802d300 con 0x7f2b28102140 2024-03-24T02:58:59.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.891+0000 7f2b267fc640 1 -- 172.21.15.17:0/3448624485 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11452 (secure 0 0 0) 0x7f2b2009b820 con 0x7f2b28102140 2024-03-24T02:58:59.891 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:58:59.891 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T02:58:36.476051+0000","modified":"2024-03-24T02:58:59.074424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24529,"mds_1":24521,"mds_2":24511,"mds_3":24493,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6841/1592961709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":1592961709},{"type":"v1","addr":"172.21.15.17:6841","nonce":1592961709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6839/2102503456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2102503456},{"type":"v1","addr":"172.21.15.19:6839","nonce":2102503456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24511,24529,24485,24493,24521]},"id":1}]} 2024-03-24T02:58:59.891 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 28 2024-03-24T02:58:59.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.893+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/3448624485 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2b100a2890 msgr2=0x7f2b100a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:59.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.893+0000 7f2b2f8f1640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2b100a2890 0x7f2b100a4d50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c02c170 tx=0x7f2b1c02df90 comp rx=0 tx=0).stop 2024-03-24T02:58:59.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.893+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/3448624485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b28102140 msgr2=0x7f2b28197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:58:59.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.893+0000 7f2b2f8f1640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b28102140 0x7f2b28197a10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000e9f0 tx=0x7f2b2000edf0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.893+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/3448624485 shutdown_connections 2024-03-24T02:58:59.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.893+0000 7f2b2f8f1640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2b100a2890 0x7f2b100a4d50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.893+0000 7f2b2f8f1640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b281073a0 0x7f2b2819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.893+0000 7f2b2f8f1640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b28104a70 0x7f2b28197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.893+0000 7f2b2f8f1640 1 --2- 172.21.15.17:0/3448624485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b28102140 0x7f2b28197a10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:58:59.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.893+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/3448624485 >> 172.21.15.17:0/3448624485 conn(0x7f2b280fbb50 msgr2=0x7f2b281029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:58:59.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.893+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/3448624485 shutdown_connections 2024-03-24T02:58:59.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:58:59.893+0000 7f2b2f8f1640 1 -- 172.21.15.17:0/3448624485 wait complete. 2024-03-24T02:58:59.904 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T02:59:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.007+0000 7fd87743f640 1 -- 172.21.15.17:0/2125543783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8701017b0 msgr2=0x7fd870101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.007+0000 7fd87743f640 1 --2- 172.21.15.17:0/2125543783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8701017b0 0x7fd870101c30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd860009900 tx=0x7fd86002bc30 comp rx=0 tx=0).stop 2024-03-24T02:59:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.007+0000 7fd87743f640 1 -- 172.21.15.17:0/2125543783 shutdown_connections 2024-03-24T02:59:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.007+0000 7fd87743f640 1 --2- 172.21.15.17:0/2125543783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd870102170 0x7fd870108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.007+0000 7fd87743f640 1 --2- 172.21.15.17:0/2125543783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8701017b0 0x7fd870101c30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.007+0000 7fd87743f640 1 --2- 172.21.15.17:0/2125543783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8701005c0 0x7fd8701009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.007+0000 7fd87743f640 1 -- 172.21.15.17:0/2125543783 >> 172.21.15.17:0/2125543783 conn(0x7fd8700fbb30 msgr2=0x7fd8700fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.007+0000 7fd87743f640 1 -- 172.21.15.17:0/2125543783 shutdown_connections 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.007+0000 7fd87743f640 1 -- 172.21.15.17:0/2125543783 wait complete. 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd87743f640 1 Processor -- start 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd87743f640 1 -- start start 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd87743f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8701005c0 0x7fd8701979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd87743f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8701017b0 0x7fd870197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd87743f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd870102170 0x7fd87019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd87743f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87010d360 con 0x7fd8701005c0 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd87743f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87010d1e0 con 0x7fd8701017b0 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd87743f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87010d4e0 con 0x7fd870102170 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd8751db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8701005c0 0x7fd8701979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd8751db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8701005c0 0x7fd8701979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53398/0 (socket says 172.21.15.17:53398) 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd8751db640 1 -- 172.21.15.17:0/890970100 learned_addr learned my addr 172.21.15.17:0/890970100 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd8749da640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8701017b0 0x7fd870197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.008+0000 7fd8759dc640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd870102170 0x7fd87019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.009+0000 7fd8751db640 1 -- 172.21.15.17:0/890970100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd870102170 msgr2=0x7fd87019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.009+0000 7fd8751db640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd870102170 0x7fd87019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.009+0000 7fd8751db640 1 -- 172.21.15.17:0/890970100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8701017b0 msgr2=0x7fd870197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.009+0000 7fd8751db640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8701017b0 0x7fd870197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.009+0000 7fd8751db640 1 -- 172.21.15.17:0/890970100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87019f5d0 con 0x7fd8701005c0 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.009+0000 7fd8751db640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8701005c0 0x7fd8701979b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00e9c0 tx=0x7fd86c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.009+0000 7fd8667fc640 1 -- 172.21.15.17:0/890970100 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c00cd80 con 0x7fd8701005c0 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.009+0000 7fd8667fc640 1 -- 172.21.15.17:0/890970100 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd86c0035f0 con 0x7fd8701005c0 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.009+0000 7fd87743f640 1 -- 172.21.15.17:0/890970100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87019f8c0 con 0x7fd8701005c0 2024-03-24T02:59:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.009+0000 7fd8667fc640 1 -- 172.21.15.17:0/890970100 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c010690 con 0x7fd8701005c0 2024-03-24T02:59:00.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.009+0000 7fd87743f640 1 -- 172.21.15.17:0/890970100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87019fde0 con 0x7fd8701005c0 2024-03-24T02:59:00.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.010+0000 7fd87743f640 1 -- 172.21.15.17:0/890970100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8701078e0 con 0x7fd8701005c0 2024-03-24T02:59:00.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.011+0000 7fd8667fc640 1 -- 172.21.15.17:0/890970100 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fd86c0037a0 con 0x7fd8701005c0 2024-03-24T02:59:00.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.012+0000 7fd8667fc640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd85c0a2890 0x7fd85c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:00.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.012+0000 7fd8749da640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd85c0a2890 0x7fd85c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:00.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.012+0000 7fd8667fc640 1 -- 172.21.15.17:0/890970100 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd86c0cf320 con 0x7fd8701005c0 2024-03-24T02:59:00.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.012+0000 7fd8749da640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd85c0a2890 0x7fd85c0a4d50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd86002c7b0 tx=0x7fd860005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.015+0000 7fd8667fc640 1 -- 172.21.15.17:0/890970100 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd86c09c050 con 0x7fd8701005c0 2024-03-24T02:59:00.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.203+0000 7fd87743f640 1 -- 172.21.15.17:0/890970100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd87002d070 con 0x7fd8701005c0 2024-03-24T02:59:00.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.204+0000 7fd8667fc640 1 -- 172.21.15.17:0/890970100 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24100 (secure 0 0 0) 0x7fd86c09b950 con 0x7fd8701005c0 2024-03-24T02:59:00.204 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:59:00.204 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:58:41.346962+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:59:00.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.206+0000 7fd87743f640 1 -- 172.21.15.17:0/890970100 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd85c0a2890 msgr2=0x7fd85c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.206+0000 7fd87743f640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd85c0a2890 0x7fd85c0a4d50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd86002c7b0 tx=0x7fd860005fd0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.206+0000 7fd87743f640 1 -- 172.21.15.17:0/890970100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8701005c0 msgr2=0x7fd8701979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.206+0000 7fd87743f640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8701005c0 0x7fd8701979b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00e9c0 tx=0x7fd86c00ee80 comp rx=0 tx=0).stop 2024-03-24T02:59:00.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.206+0000 7fd87743f640 1 -- 172.21.15.17:0/890970100 shutdown_connections 2024-03-24T02:59:00.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.206+0000 7fd87743f640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd85c0a2890 0x7fd85c0a4d50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.206+0000 7fd87743f640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd870102170 0x7fd87019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.206+0000 7fd87743f640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8701017b0 0x7fd870197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.206+0000 7fd87743f640 1 --2- 172.21.15.17:0/890970100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8701005c0 0x7fd8701979b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.206+0000 7fd87743f640 1 -- 172.21.15.17:0/890970100 >> 172.21.15.17:0/890970100 conn(0x7fd8700fbb30 msgr2=0x7fd870104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:00.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:00 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] up:active 2024-03-24T02:59:00.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:00 smithi017 ceph-mon[31926]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-24T02:59:00.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:00 smithi017 ceph-mon[31926]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-24T02:59:00.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:00 smithi017 ceph-mon[31926]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-24T02:59:00.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:00 smithi017 ceph-mon[31926]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:59:00.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:00 smithi017 ceph-mon[31926]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:59:00.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:00 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3448624485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:59:00.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:00 smithi017 ceph-mon[31926]: pgmap v304: 129 pgs: 129 active+clean; 587 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-03-24T02:59:00.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.207+0000 7fd87743f640 1 -- 172.21.15.17:0/890970100 shutdown_connections 2024-03-24T02:59:00.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.207+0000 7fd87743f640 1 -- 172.21.15.17:0/890970100 wait complete. 2024-03-24T02:59:00.217 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-24T02:59:00.217 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-24T02:59:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.321+0000 7f94f1236640 1 -- 172.21.15.17:0/1524560687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94ec1005e0 msgr2=0x7f94ec100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.321+0000 7f94f1236640 1 --2- 172.21.15.17:0/1524560687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94ec1005e0 0x7f94ec100a00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009900 tx=0x7f94dc02bc30 comp rx=0 tx=0).stop 2024-03-24T02:59:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.321+0000 7f94f1236640 1 -- 172.21.15.17:0/1524560687 shutdown_connections 2024-03-24T02:59:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.321+0000 7f94f1236640 1 --2- 172.21.15.17:0/1524560687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ec102190 0x7f94ec108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.321+0000 7f94f1236640 1 --2- 172.21.15.17:0/1524560687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94ec1017d0 0x7f94ec101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.321+0000 7f94f1236640 1 --2- 172.21.15.17:0/1524560687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94ec1005e0 0x7f94ec100a00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.321+0000 7f94f1236640 1 -- 172.21.15.17:0/1524560687 >> 172.21.15.17:0/1524560687 conn(0x7f94ec0fbb50 msgr2=0x7f94ec0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.321+0000 7f94f1236640 1 -- 172.21.15.17:0/1524560687 shutdown_connections 2024-03-24T02:59:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.321+0000 7f94f1236640 1 -- 172.21.15.17:0/1524560687 wait complete. 2024-03-24T02:59:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94f1236640 1 Processor -- start 2024-03-24T02:59:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94f1236640 1 -- start start 2024-03-24T02:59:00.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94f1236640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94ec1005e0 0x7f94ec06a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:00.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94f1236640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94ec1017d0 0x7f94ec19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:00.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94f1236640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ec102190 0x7f94ec19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:00.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94f1236640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec10c250 con 0x7f94ec1005e0 2024-03-24T02:59:00.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94f1236640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec10c0d0 con 0x7f94ec1017d0 2024-03-24T02:59:00.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94f1236640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec10c3d0 con 0x7f94ec102190 2024-03-24T02:59:00.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94ead9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94ec1005e0 0x7f94ec06a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:00.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94ead9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94ec1005e0 0x7f94ec06a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53416/0 (socket says 172.21.15.17:53416) 2024-03-24T02:59:00.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94ead9d640 1 -- 172.21.15.17:0/3425892609 learned_addr learned my addr 172.21.15.17:0/3425892609 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:00.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94e3fff640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94ec1017d0 0x7f94ec19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:00.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94eb59e640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ec102190 0x7f94ec19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94ead9d640 1 -- 172.21.15.17:0/3425892609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ec102190 msgr2=0x7f94ec19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94ead9d640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ec102190 0x7f94ec19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94ead9d640 1 -- 172.21.15.17:0/3425892609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94ec1017d0 msgr2=0x7f94ec19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94ead9d640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94ec1017d0 0x7f94ec19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94ead9d640 1 -- 172.21.15.17:0/3425892609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec19f600 con 0x7f94ec1005e0 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94eb59e640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ec102190 0x7f94ec19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.322+0000 7f94e3fff640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94ec1017d0 0x7f94ec19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.323+0000 7f94ead9d640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94ec1005e0 0x7f94ec06a860 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94dc000c00 tx=0x7f94dc02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.323+0000 7f94e8d99640 1 -- 172.21.15.17:0/3425892609 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc041070 con 0x7f94ec1005e0 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.323+0000 7f94e8d99640 1 -- 172.21.15.17:0/3425892609 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94dc004450 con 0x7f94ec1005e0 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.323+0000 7f94f1236640 1 -- 172.21.15.17:0/3425892609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec19f890 con 0x7f94ec1005e0 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.323+0000 7f94e8d99640 1 -- 172.21.15.17:0/3425892609 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc034440 con 0x7f94ec1005e0 2024-03-24T02:59:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.323+0000 7f94f1236640 1 -- 172.21.15.17:0/3425892609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec19fd80 con 0x7f94ec1005e0 2024-03-24T02:59:00.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.324+0000 7f94f1236640 1 -- 172.21.15.17:0/3425892609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec107900 con 0x7f94ec1005e0 2024-03-24T02:59:00.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.325+0000 7f94e8d99640 1 -- 172.21.15.17:0/3425892609 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f94dc044070 con 0x7f94ec1005e0 2024-03-24T02:59:00.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.326+0000 7f94e8d99640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f94d80a2890 0x7f94d80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:00.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.326+0000 7f94e3fff640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f94d80a2890 0x7f94d80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:00.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.326+0000 7f94e8d99640 1 -- 172.21.15.17:0/3425892609 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f94dc0f0a50 con 0x7f94ec1005e0 2024-03-24T02:59:00.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.326+0000 7f94e3fff640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f94d80a2890 0x7f94d80a4d50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f94ec06a5d0 tx=0x7f94d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:00.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.329+0000 7f94e8d99640 1 -- 172.21.15.17:0/3425892609 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f94dc0bd240 con 0x7f94ec1005e0 2024-03-24T02:59:00.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.510+0000 7f94f1236640 1 -- 172.21.15.17:0/3425892609 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94ec061ae0 con 0x7f94d80a2890 2024-03-24T02:59:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.520+0000 7f94e8d99640 1 -- 172.21.15.17:0/3425892609 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340401 (secure 0 0 0) 0x7f94ec061ae0 con 0x7f94d80a2890 2024-03-24T02:59:00.526 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:59:00.528 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-24T02:59:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.524+0000 7f94f1236640 1 -- 172.21.15.17:0/3425892609 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f94d80a2890 msgr2=0x7f94d80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.524+0000 7f94f1236640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f94d80a2890 0x7f94d80a4d50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f94ec06a5d0 tx=0x7f94d4005f90 comp rx=0 tx=0).stop 2024-03-24T02:59:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.524+0000 7f94f1236640 1 -- 172.21.15.17:0/3425892609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94ec1005e0 msgr2=0x7f94ec06a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.524+0000 7f94f1236640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94ec1005e0 0x7f94ec06a860 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94dc000c00 tx=0x7f94dc02c640 comp rx=0 tx=0).stop 2024-03-24T02:59:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.524+0000 7f94f1236640 1 -- 172.21.15.17:0/3425892609 shutdown_connections 2024-03-24T02:59:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.524+0000 7f94f1236640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f94d80a2890 0x7f94d80a4d50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.524+0000 7f94f1236640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ec102190 0x7f94ec19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.524+0000 7f94f1236640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94ec1017d0 0x7f94ec19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.524+0000 7f94f1236640 1 --2- 172.21.15.17:0/3425892609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94ec1005e0 0x7f94ec06a860 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.524+0000 7f94f1236640 1 -- 172.21.15.17:0/3425892609 >> 172.21.15.17:0/3425892609 conn(0x7f94ec0fbb50 msgr2=0x7f94ec104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.524+0000 7f94f1236640 1 -- 172.21.15.17:0/3425892609 shutdown_connections 2024-03-24T02:59:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.524+0000 7f94f1236640 1 -- 172.21.15.17:0/3425892609 wait complete. 2024-03-24T02:59:00.534 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-03-24T02:58:59.950482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600774,"num_objects":82,"num_object_clones":0,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":152,"num_read_kb":132,"num_write":350,"num_write_kb":2997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":286,"ondisk_log_size":286,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404356,"kb_used_data":13596,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537468,"statfs":{"total":1151940427776,"available":1151526367232,"internally_reserved":0,"allocated":13922304,"data_stored":7557817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7996,"num_objects":58,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":124,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003650"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527985+0000","last_change":"2024-03-24T02:58:36.978486+0000","last_active":"2024-03-24T02:58:43.527985+0000","last_peered":"2024-03-24T02:58:43.527985+0000","last_clean":"2024-03-24T02:58:43.527985+0000","last_became_active":"2024-03-24T02:58:36.976937+0000","last_became_peered":"2024-03-24T02:58:36.976937+0000","last_unstale":"2024-03-24T02:58:43.527985+0000","last_undegraded":"2024-03-24T02:58:43.527985+0000","last_fullsized":"2024-03-24T02:58:43.527985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:25:09.439179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528138+0000","last_change":"2024-03-24T02:58:36.983050+0000","last_active":"2024-03-24T02:58:43.528138+0000","last_peered":"2024-03-24T02:58:43.528138+0000","last_clean":"2024-03-24T02:58:43.528138+0000","last_became_active":"2024-03-24T02:58:36.982741+0000","last_became_peered":"2024-03-24T02:58:36.982741+0000","last_unstale":"2024-03-24T02:58:43.528138+0000","last_undegraded":"2024-03-24T02:58:43.528138+0000","last_fullsized":"2024-03-24T02:58:43.528138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:26:35.938559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527776+0000","last_change":"2024-03-24T02:58:36.976313+0000","last_active":"2024-03-24T02:58:43.527776+0000","last_peered":"2024-03-24T02:58:43.527776+0000","last_clean":"2024-03-24T02:58:43.527776+0000","last_became_active":"2024-03-24T02:58:36.975087+0000","last_became_peered":"2024-03-24T02:58:36.975087+0000","last_unstale":"2024-03-24T02:58:43.527776+0000","last_undegraded":"2024-03-24T02:58:43.527776+0000","last_fullsized":"2024-03-24T02:58:43.527776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:22:31.476824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528524+0000","last_change":"2024-03-24T02:58:36.977632+0000","last_active":"2024-03-24T02:58:43.528524+0000","last_peered":"2024-03-24T02:58:43.528524+0000","last_clean":"2024-03-24T02:58:43.528524+0000","last_became_active":"2024-03-24T02:58:36.977111+0000","last_became_peered":"2024-03-24T02:58:36.977111+0000","last_unstale":"2024-03-24T02:58:43.528524+0000","last_undegraded":"2024-03-24T02:58:43.528524+0000","last_fullsized":"2024-03-24T02:58:43.528524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:47:07.874538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986247+0000","last_change":"2024-03-24T02:58:36.967543+0000","last_active":"2024-03-24T02:58:41.986247+0000","last_peered":"2024-03-24T02:58:41.986247+0000","last_clean":"2024-03-24T02:58:41.986247+0000","last_became_active":"2024-03-24T02:58:36.967108+0000","last_became_peered":"2024-03-24T02:58:36.967108+0000","last_unstale":"2024-03-24T02:58:41.986247+0000","last_undegraded":"2024-03-24T02:58:41.986247+0000","last_fullsized":"2024-03-24T02:58:41.986247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:38:46.280891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.989906+0000","last_change":"2024-03-24T02:58:36.977777+0000","last_active":"2024-03-24T02:58:42.989906+0000","last_peered":"2024-03-24T02:58:42.989906+0000","last_clean":"2024-03-24T02:58:42.989906+0000","last_became_active":"2024-03-24T02:58:36.976615+0000","last_became_peered":"2024-03-24T02:58:36.976615+0000","last_unstale":"2024-03-24T02:58:42.989906+0000","last_undegraded":"2024-03-24T02:58:42.989906+0000","last_fullsized":"2024-03-24T02:58:42.989906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:13:15.149963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.989245+0000","last_change":"2024-03-24T02:58:36.990832+0000","last_active":"2024-03-24T02:58:41.989245+0000","last_peered":"2024-03-24T02:58:41.989245+0000","last_clean":"2024-03-24T02:58:41.989245+0000","last_became_active":"2024-03-24T02:58:36.990395+0000","last_became_peered":"2024-03-24T02:58:36.990395+0000","last_unstale":"2024-03-24T02:58:41.989245+0000","last_undegraded":"2024-03-24T02:58:41.989245+0000","last_fullsized":"2024-03-24T02:58:41.989245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:34.915192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.026791+0000","last_change":"2024-03-24T02:58:36.980829+0000","last_active":"2024-03-24T02:58:43.026791+0000","last_peered":"2024-03-24T02:58:43.026791+0000","last_clean":"2024-03-24T02:58:43.026791+0000","last_became_active":"2024-03-24T02:58:36.978641+0000","last_became_peered":"2024-03-24T02:58:36.978641+0000","last_unstale":"2024-03-24T02:58:43.026791+0000","last_undegraded":"2024-03-24T02:58:43.026791+0000","last_fullsized":"2024-03-24T02:58:43.026791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:56:29.948080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.529167+0000","last_change":"2024-03-24T02:58:36.989173+0000","last_active":"2024-03-24T02:58:43.529167+0000","last_peered":"2024-03-24T02:58:43.529167+0000","last_clean":"2024-03-24T02:58:43.529167+0000","last_became_active":"2024-03-24T02:58:36.988771+0000","last_became_peered":"2024-03-24T02:58:36.988771+0000","last_unstale":"2024-03-24T02:58:43.529167+0000","last_undegraded":"2024-03-24T02:58:43.529167+0000","last_fullsized":"2024-03-24T02:58:43.529167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:51:01.790952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.990653+0000","last_change":"2024-03-24T02:58:36.978505+0000","last_active":"2024-03-24T02:58:42.990653+0000","last_peered":"2024-03-24T02:58:42.990653+0000","last_clean":"2024-03-24T02:58:42.990653+0000","last_became_active":"2024-03-24T02:58:36.978145+0000","last_became_peered":"2024-03-24T02:58:36.978145+0000","last_unstale":"2024-03-24T02:58:42.990653+0000","last_undegraded":"2024-03-24T02:58:42.990653+0000","last_fullsized":"2024-03-24T02:58:42.990653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:06:41.874675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.989375+0000","last_change":"2024-03-24T02:58:36.984134+0000","last_active":"2024-03-24T02:58:41.989375+0000","last_peered":"2024-03-24T02:58:41.989375+0000","last_clean":"2024-03-24T02:58:41.989375+0000","last_became_active":"2024-03-24T02:58:36.981037+0000","last_became_peered":"2024-03-24T02:58:36.981037+0000","last_unstale":"2024-03-24T02:58:41.989375+0000","last_undegraded":"2024-03-24T02:58:41.989375+0000","last_fullsized":"2024-03-24T02:58:41.989375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:06:40.023840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528397+0000","last_change":"2024-03-24T02:58:36.979852+0000","last_active":"2024-03-24T02:58:43.528397+0000","last_peered":"2024-03-24T02:58:43.528397+0000","last_clean":"2024-03-24T02:58:43.528397+0000","last_became_active":"2024-03-24T02:58:36.979506+0000","last_became_peered":"2024-03-24T02:58:36.979506+0000","last_unstale":"2024-03-24T02:58:43.528397+0000","last_undegraded":"2024-03-24T02:58:43.528397+0000","last_fullsized":"2024-03-24T02:58:43.528397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:01:27.493694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527939+0000","last_change":"2024-03-24T02:58:36.977018+0000","last_active":"2024-03-24T02:58:43.527939+0000","last_peered":"2024-03-24T02:58:43.527939+0000","last_clean":"2024-03-24T02:58:43.527939+0000","last_became_active":"2024-03-24T02:58:36.976669+0000","last_became_peered":"2024-03-24T02:58:36.976669+0000","last_unstale":"2024-03-24T02:58:43.527939+0000","last_undegraded":"2024-03-24T02:58:43.527939+0000","last_fullsized":"2024-03-24T02:58:43.527939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:23:15.049093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527483+0000","last_change":"2024-03-24T02:58:36.973200+0000","last_active":"2024-03-24T02:58:43.527483+0000","last_peered":"2024-03-24T02:58:43.527483+0000","last_clean":"2024-03-24T02:58:43.527483+0000","last_became_active":"2024-03-24T02:58:36.972828+0000","last_became_peered":"2024-03-24T02:58:36.972828+0000","last_unstale":"2024-03-24T02:58:43.527483+0000","last_undegraded":"2024-03-24T02:58:43.527483+0000","last_fullsized":"2024-03-24T02:58:43.527483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:42:25.122881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527580+0000","last_change":"2024-03-24T02:58:36.977686+0000","last_active":"2024-03-24T02:58:43.527580+0000","last_peered":"2024-03-24T02:58:43.527580+0000","last_clean":"2024-03-24T02:58:43.527580+0000","last_became_active":"2024-03-24T02:58:36.976311+0000","last_became_peered":"2024-03-24T02:58:36.976311+0000","last_unstale":"2024-03-24T02:58:43.527580+0000","last_undegraded":"2024-03-24T02:58:43.527580+0000","last_fullsized":"2024-03-24T02:58:43.527580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:10:04.585726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528141+0000","last_change":"2024-03-24T02:58:36.985336+0000","last_active":"2024-03-24T02:58:43.528141+0000","last_peered":"2024-03-24T02:58:43.528141+0000","last_clean":"2024-03-24T02:58:43.528141+0000","last_became_active":"2024-03-24T02:58:36.984842+0000","last_became_peered":"2024-03-24T02:58:36.984842+0000","last_unstale":"2024-03-24T02:58:43.528141+0000","last_undegraded":"2024-03-24T02:58:43.528141+0000","last_fullsized":"2024-03-24T02:58:43.528141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:52:41.819132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527858+0000","last_change":"2024-03-24T02:58:36.978310+0000","last_active":"2024-03-24T02:58:43.527858+0000","last_peered":"2024-03-24T02:58:43.527858+0000","last_clean":"2024-03-24T02:58:43.527858+0000","last_became_active":"2024-03-24T02:58:36.977396+0000","last_became_peered":"2024-03-24T02:58:36.977396+0000","last_unstale":"2024-03-24T02:58:43.527858+0000","last_undegraded":"2024-03-24T02:58:43.527858+0000","last_fullsized":"2024-03-24T02:58:43.527858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:37:21.200591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986366+0000","last_change":"2024-03-24T02:58:36.979514+0000","last_active":"2024-03-24T02:58:41.986366+0000","last_peered":"2024-03-24T02:58:41.986366+0000","last_clean":"2024-03-24T02:58:41.986366+0000","last_became_active":"2024-03-24T02:58:36.979155+0000","last_became_peered":"2024-03-24T02:58:36.979155+0000","last_unstale":"2024-03-24T02:58:41.986366+0000","last_undegraded":"2024-03-24T02:58:41.986366+0000","last_fullsized":"2024-03-24T02:58:41.986366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:49:37.142195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575668+0000","last_change":"2024-03-24T02:58:36.971546+0000","last_active":"2024-03-24T02:58:42.575668+0000","last_peered":"2024-03-24T02:58:42.575668+0000","last_clean":"2024-03-24T02:58:42.575668+0000","last_became_active":"2024-03-24T02:58:36.971210+0000","last_became_peered":"2024-03-24T02:58:36.971210+0000","last_unstale":"2024-03-24T02:58:42.575668+0000","last_undegraded":"2024-03-24T02:58:42.575668+0000","last_fullsized":"2024-03-24T02:58:42.575668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:35:21.274898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575518+0000","last_change":"2024-03-24T02:58:40.994331+0000","last_active":"2024-03-24T02:58:42.575518+0000","last_peered":"2024-03-24T02:58:42.575518+0000","last_clean":"2024-03-24T02:58:42.575518+0000","last_became_active":"2024-03-24T02:58:40.993880+0000","last_became_peered":"2024-03-24T02:58:40.993880+0000","last_unstale":"2024-03-24T02:58:42.575518+0000","last_undegraded":"2024-03-24T02:58:42.575518+0000","last_fullsized":"2024-03-24T02:58:42.575518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:58:46.876872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:56.059840+0000","last_change":"2024-03-24T02:58:36.132975+0000","last_active":"2024-03-24T02:58:56.059840+0000","last_peered":"2024-03-24T02:58:56.059840+0000","last_clean":"2024-03-24T02:58:56.059840+0000","last_became_active":"2024-03-24T02:58:36.132640+0000","last_became_peered":"2024-03-24T02:58:36.132640+0000","last_unstale":"2024-03-24T02:58:56.059840+0000","last_undegraded":"2024-03-24T02:58:56.059840+0000","last_fullsized":"2024-03-24T02:58:56.059840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:37:03.765773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.027263+0000","last_change":"2024-03-24T02:58:36.977283+0000","last_active":"2024-03-24T02:58:43.027263+0000","last_peered":"2024-03-24T02:58:43.027263+0000","last_clean":"2024-03-24T02:58:43.027263+0000","last_became_active":"2024-03-24T02:58:36.976968+0000","last_became_peered":"2024-03-24T02:58:36.976968+0000","last_unstale":"2024-03-24T02:58:43.027263+0000","last_undegraded":"2024-03-24T02:58:43.027263+0000","last_fullsized":"2024-03-24T02:58:43.027263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:28:12.115521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527852+0000","last_change":"2024-03-24T02:58:40.997896+0000","last_active":"2024-03-24T02:58:43.527852+0000","last_peered":"2024-03-24T02:58:43.527852+0000","last_clean":"2024-03-24T02:58:43.527852+0000","last_became_active":"2024-03-24T02:58:40.996845+0000","last_became_peered":"2024-03-24T02:58:40.996845+0000","last_unstale":"2024-03-24T02:58:43.527852+0000","last_undegraded":"2024-03-24T02:58:43.527852+0000","last_fullsized":"2024-03-24T02:58:43.527852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:51:17.031332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.027447+0000","last_change":"2024-03-24T02:58:36.139069+0000","last_active":"2024-03-24T02:58:43.027447+0000","last_peered":"2024-03-24T02:58:43.027447+0000","last_clean":"2024-03-24T02:58:43.027447+0000","last_became_active":"2024-03-24T02:58:36.138571+0000","last_became_peered":"2024-03-24T02:58:36.138571+0000","last_unstale":"2024-03-24T02:58:43.027447+0000","last_undegraded":"2024-03-24T02:58:43.027447+0000","last_fullsized":"2024-03-24T02:58:43.027447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:17:44.510904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528182+0000","last_change":"2024-03-24T02:58:36.976727+0000","last_active":"2024-03-24T02:58:43.528182+0000","last_peered":"2024-03-24T02:58:43.528182+0000","last_clean":"2024-03-24T02:58:43.528182+0000","last_became_active":"2024-03-24T02:58:36.976076+0000","last_became_peered":"2024-03-24T02:58:36.976076+0000","last_unstale":"2024-03-24T02:58:43.528182+0000","last_undegraded":"2024-03-24T02:58:43.528182+0000","last_fullsized":"2024-03-24T02:58:43.528182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:33:03.490882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528014+0000","last_change":"2024-03-24T02:58:41.003025+0000","last_active":"2024-03-24T02:58:43.528014+0000","last_peered":"2024-03-24T02:58:43.528014+0000","last_clean":"2024-03-24T02:58:43.528014+0000","last_became_active":"2024-03-24T02:58:41.002270+0000","last_became_peered":"2024-03-24T02:58:41.002270+0000","last_unstale":"2024-03-24T02:58:43.528014+0000","last_undegraded":"2024-03-24T02:58:43.528014+0000","last_fullsized":"2024-03-24T02:58:43.528014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:31:36.390191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986643+0000","last_change":"2024-03-24T02:58:36.135836+0000","last_active":"2024-03-24T02:58:41.986643+0000","last_peered":"2024-03-24T02:58:41.986643+0000","last_clean":"2024-03-24T02:58:41.986643+0000","last_became_active":"2024-03-24T02:58:36.135476+0000","last_became_peered":"2024-03-24T02:58:36.135476+0000","last_unstale":"2024-03-24T02:58:41.986643+0000","last_undegraded":"2024-03-24T02:58:41.986643+0000","last_fullsized":"2024-03-24T02:58:41.986643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:30:14.058373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528539+0000","last_change":"2024-03-24T02:58:36.981600+0000","last_active":"2024-03-24T02:58:43.528539+0000","last_peered":"2024-03-24T02:58:43.528539+0000","last_clean":"2024-03-24T02:58:43.528539+0000","last_became_active":"2024-03-24T02:58:36.981222+0000","last_became_peered":"2024-03-24T02:58:36.981222+0000","last_unstale":"2024-03-24T02:58:43.528539+0000","last_undegraded":"2024-03-24T02:58:43.528539+0000","last_fullsized":"2024-03-24T02:58:43.528539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:11:33.771157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528011+0000","last_change":"2024-03-24T02:58:40.996838+0000","last_active":"2024-03-24T02:58:43.528011+0000","last_peered":"2024-03-24T02:58:43.528011+0000","last_clean":"2024-03-24T02:58:43.528011+0000","last_became_active":"2024-03-24T02:58:40.996320+0000","last_became_peered":"2024-03-24T02:58:40.996320+0000","last_unstale":"2024-03-24T02:58:43.528011+0000","last_undegraded":"2024-03-24T02:58:43.528011+0000","last_fullsized":"2024-03-24T02:58:43.528011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:11:34.124402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986402+0000","last_change":"2024-03-24T02:58:36.132406+0000","last_active":"2024-03-24T02:58:41.986402+0000","last_peered":"2024-03-24T02:58:41.986402+0000","last_clean":"2024-03-24T02:58:41.986402+0000","last_became_active":"2024-03-24T02:58:36.132044+0000","last_became_peered":"2024-03-24T02:58:36.132044+0000","last_unstale":"2024-03-24T02:58:41.986402+0000","last_undegraded":"2024-03-24T02:58:41.986402+0000","last_fullsized":"2024-03-24T02:58:41.986402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:35:53.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528508+0000","last_change":"2024-03-24T02:58:36.977989+0000","last_active":"2024-03-24T02:58:43.528508+0000","last_peered":"2024-03-24T02:58:43.528508+0000","last_clean":"2024-03-24T02:58:43.528508+0000","last_became_active":"2024-03-24T02:58:36.977653+0000","last_became_peered":"2024-03-24T02:58:36.977653+0000","last_unstale":"2024-03-24T02:58:43.528508+0000","last_undegraded":"2024-03-24T02:58:43.528508+0000","last_fullsized":"2024-03-24T02:58:43.528508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:21:18.364057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575573+0000","last_change":"2024-03-24T02:58:40.989082+0000","last_active":"2024-03-24T02:58:42.575573+0000","last_peered":"2024-03-24T02:58:42.575573+0000","last_clean":"2024-03-24T02:58:42.575573+0000","last_became_active":"2024-03-24T02:58:40.988731+0000","last_became_peered":"2024-03-24T02:58:40.988731+0000","last_unstale":"2024-03-24T02:58:42.575573+0000","last_undegraded":"2024-03-24T02:58:42.575573+0000","last_fullsized":"2024-03-24T02:58:42.575573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:34:39.489446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.050467+0000","last_change":"2024-03-24T02:58:36.135785+0000","last_active":"2024-03-24T02:58:55.050467+0000","last_peered":"2024-03-24T02:58:55.050467+0000","last_clean":"2024-03-24T02:58:55.050467+0000","last_became_active":"2024-03-24T02:58:36.135470+0000","last_became_peered":"2024-03-24T02:58:36.135470+0000","last_unstale":"2024-03-24T02:58:55.050467+0000","last_undegraded":"2024-03-24T02:58:55.050467+0000","last_fullsized":"2024-03-24T02:58:55.050467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:41:26.529773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986204+0000","last_change":"2024-03-24T02:58:36.978241+0000","last_active":"2024-03-24T02:58:41.986204+0000","last_peered":"2024-03-24T02:58:41.986204+0000","last_clean":"2024-03-24T02:58:41.986204+0000","last_became_active":"2024-03-24T02:58:36.977879+0000","last_became_peered":"2024-03-24T02:58:36.977879+0000","last_unstale":"2024-03-24T02:58:41.986204+0000","last_undegraded":"2024-03-24T02:58:41.986204+0000","last_fullsized":"2024-03-24T02:58:41.986204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:44:41.190736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575612+0000","last_change":"2024-03-24T02:58:41.004890+0000","last_active":"2024-03-24T02:58:42.575612+0000","last_peered":"2024-03-24T02:58:42.575612+0000","last_clean":"2024-03-24T02:58:42.575612+0000","last_became_active":"2024-03-24T02:58:41.004428+0000","last_became_peered":"2024-03-24T02:58:41.004428+0000","last_unstale":"2024-03-24T02:58:42.575612+0000","last_undegraded":"2024-03-24T02:58:42.575612+0000","last_fullsized":"2024-03-24T02:58:42.575612+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:43:29.488561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:57.066003+0000","last_change":"2024-03-24T02:58:36.132242+0000","last_active":"2024-03-24T02:58:57.066003+0000","last_peered":"2024-03-24T02:58:57.066003+0000","last_clean":"2024-03-24T02:58:57.066003+0000","last_became_active":"2024-03-24T02:58:36.131792+0000","last_became_peered":"2024-03-24T02:58:36.131792+0000","last_unstale":"2024-03-24T02:58:57.066003+0000","last_undegraded":"2024-03-24T02:58:57.066003+0000","last_fullsized":"2024-03-24T02:58:57.066003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:09:16.751023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528421+0000","last_change":"2024-03-24T02:58:36.977306+0000","last_active":"2024-03-24T02:58:43.528421+0000","last_peered":"2024-03-24T02:58:43.528421+0000","last_clean":"2024-03-24T02:58:43.528421+0000","last_became_active":"2024-03-24T02:58:36.976955+0000","last_became_peered":"2024-03-24T02:58:36.976955+0000","last_unstale":"2024-03-24T02:58:43.528421+0000","last_undegraded":"2024-03-24T02:58:43.528421+0000","last_fullsized":"2024-03-24T02:58:43.528421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:09:56.568781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527915+0000","last_change":"2024-03-24T02:58:40.999977+0000","last_active":"2024-03-24T02:58:43.527915+0000","last_peered":"2024-03-24T02:58:43.527915+0000","last_clean":"2024-03-24T02:58:43.527915+0000","last_became_active":"2024-03-24T02:58:40.999564+0000","last_became_peered":"2024-03-24T02:58:40.999564+0000","last_unstale":"2024-03-24T02:58:43.527915+0000","last_undegraded":"2024-03-24T02:58:43.527915+0000","last_fullsized":"2024-03-24T02:58:43.527915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:16:58.022961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:56.058808+0000","last_change":"2024-03-24T02:58:36.123622+0000","last_active":"2024-03-24T02:58:56.058808+0000","last_peered":"2024-03-24T02:58:56.058808+0000","last_clean":"2024-03-24T02:58:56.058808+0000","last_became_active":"2024-03-24T02:58:36.123221+0000","last_became_peered":"2024-03-24T02:58:36.123221+0000","last_unstale":"2024-03-24T02:58:56.058808+0000","last_undegraded":"2024-03-24T02:58:56.058808+0000","last_fullsized":"2024-03-24T02:58:56.058808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:05:26.781268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.026678+0000","last_change":"2024-03-24T02:58:36.983446+0000","last_active":"2024-03-24T02:58:43.026678+0000","last_peered":"2024-03-24T02:58:43.026678+0000","last_clean":"2024-03-24T02:58:43.026678+0000","last_became_active":"2024-03-24T02:58:36.981401+0000","last_became_peered":"2024-03-24T02:58:36.981401+0000","last_unstale":"2024-03-24T02:58:43.026678+0000","last_undegraded":"2024-03-24T02:58:43.026678+0000","last_fullsized":"2024-03-24T02:58:43.026678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:12:42.836678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528020+0000","last_change":"2024-03-24T02:58:40.999348+0000","last_active":"2024-03-24T02:58:43.528020+0000","last_peered":"2024-03-24T02:58:43.528020+0000","last_clean":"2024-03-24T02:58:43.528020+0000","last_became_active":"2024-03-24T02:58:40.998932+0000","last_became_peered":"2024-03-24T02:58:40.998932+0000","last_unstale":"2024-03-24T02:58:43.528020+0000","last_undegraded":"2024-03-24T02:58:43.528020+0000","last_fullsized":"2024-03-24T02:58:43.528020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:24:29.294464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:57.064801+0000","last_change":"2024-03-24T02:58:36.137640+0000","last_active":"2024-03-24T02:58:57.064801+0000","last_peered":"2024-03-24T02:58:57.064801+0000","last_clean":"2024-03-24T02:58:57.064801+0000","last_became_active":"2024-03-24T02:58:36.137237+0000","last_became_peered":"2024-03-24T02:58:36.137237+0000","last_unstale":"2024-03-24T02:58:57.064801+0000","last_undegraded":"2024-03-24T02:58:57.064801+0000","last_fullsized":"2024-03-24T02:58:57.064801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:52:52.246968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.985962+0000","last_change":"2024-03-24T02:58:36.971696+0000","last_active":"2024-03-24T02:58:41.985962+0000","last_peered":"2024-03-24T02:58:41.985962+0000","last_clean":"2024-03-24T02:58:41.985962+0000","last_became_active":"2024-03-24T02:58:36.971318+0000","last_became_peered":"2024-03-24T02:58:36.971318+0000","last_unstale":"2024-03-24T02:58:41.985962+0000","last_undegraded":"2024-03-24T02:58:41.985962+0000","last_fullsized":"2024-03-24T02:58:41.985962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:01:30.976284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.026757+0000","last_change":"2024-03-24T02:58:40.995887+0000","last_active":"2024-03-24T02:58:43.026757+0000","last_peered":"2024-03-24T02:58:43.026757+0000","last_clean":"2024-03-24T02:58:43.026757+0000","last_became_active":"2024-03-24T02:58:40.995242+0000","last_became_peered":"2024-03-24T02:58:40.995242+0000","last_unstale":"2024-03-24T02:58:43.026757+0000","last_undegraded":"2024-03-24T02:58:43.026757+0000","last_fullsized":"2024-03-24T02:58:43.026757+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:33:40.376374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.050493+0000","last_change":"2024-03-24T02:58:36.128252+0000","last_active":"2024-03-24T02:58:55.050493+0000","last_peered":"2024-03-24T02:58:55.050493+0000","last_clean":"2024-03-24T02:58:55.050493+0000","last_became_active":"2024-03-24T02:58:36.127888+0000","last_became_peered":"2024-03-24T02:58:36.127888+0000","last_unstale":"2024-03-24T02:58:55.050493+0000","last_undegraded":"2024-03-24T02:58:55.050493+0000","last_fullsized":"2024-03-24T02:58:55.050493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:31:39.341123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.529063+0000","last_change":"2024-03-24T02:58:36.983836+0000","last_active":"2024-03-24T02:58:43.529063+0000","last_peered":"2024-03-24T02:58:43.529063+0000","last_clean":"2024-03-24T02:58:43.529063+0000","last_became_active":"2024-03-24T02:58:36.983481+0000","last_became_peered":"2024-03-24T02:58:36.983481+0000","last_unstale":"2024-03-24T02:58:43.529063+0000","last_undegraded":"2024-03-24T02:58:43.529063+0000","last_fullsized":"2024-03-24T02:58:43.529063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:01:01.707796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.985994+0000","last_change":"2024-03-24T02:58:41.000176+0000","last_active":"2024-03-24T02:58:41.985994+0000","last_peered":"2024-03-24T02:58:41.985994+0000","last_clean":"2024-03-24T02:58:41.985994+0000","last_became_active":"2024-03-24T02:58:40.999561+0000","last_became_peered":"2024-03-24T02:58:40.999561+0000","last_unstale":"2024-03-24T02:58:41.985994+0000","last_undegraded":"2024-03-24T02:58:41.985994+0000","last_fullsized":"2024-03-24T02:58:41.985994+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:20:28.678497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:58.083714+0000","last_change":"2024-03-24T02:58:36.131588+0000","last_active":"2024-03-24T02:58:58.083714+0000","last_peered":"2024-03-24T02:58:58.083714+0000","last_clean":"2024-03-24T02:58:58.083714+0000","last_became_active":"2024-03-24T02:58:36.131251+0000","last_became_peered":"2024-03-24T02:58:36.131251+0000","last_unstale":"2024-03-24T02:58:58.083714+0000","last_undegraded":"2024-03-24T02:58:58.083714+0000","last_fullsized":"2024-03-24T02:58:58.083714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:03:22.978352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528073+0000","last_change":"2024-03-24T02:58:36.978566+0000","last_active":"2024-03-24T02:58:43.528073+0000","last_peered":"2024-03-24T02:58:43.528073+0000","last_clean":"2024-03-24T02:58:43.528073+0000","last_became_active":"2024-03-24T02:58:36.977950+0000","last_became_peered":"2024-03-24T02:58:36.977950+0000","last_unstale":"2024-03-24T02:58:43.528073+0000","last_undegraded":"2024-03-24T02:58:43.528073+0000","last_fullsized":"2024-03-24T02:58:43.528073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:06:39.029357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528817+0000","last_change":"2024-03-24T02:58:41.003427+0000","last_active":"2024-03-24T02:58:43.528817+0000","last_peered":"2024-03-24T02:58:43.528817+0000","last_clean":"2024-03-24T02:58:43.528817+0000","last_became_active":"2024-03-24T02:58:41.002249+0000","last_became_peered":"2024-03-24T02:58:41.002249+0000","last_unstale":"2024-03-24T02:58:43.528817+0000","last_undegraded":"2024-03-24T02:58:43.528817+0000","last_fullsized":"2024-03-24T02:58:43.528817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:39:36.587451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.049580+0000","last_change":"2024-03-24T02:58:36.132547+0000","last_active":"2024-03-24T02:58:55.049580+0000","last_peered":"2024-03-24T02:58:55.049580+0000","last_clean":"2024-03-24T02:58:55.049580+0000","last_became_active":"2024-03-24T02:58:36.132144+0000","last_became_peered":"2024-03-24T02:58:36.132144+0000","last_unstale":"2024-03-24T02:58:55.049580+0000","last_undegraded":"2024-03-24T02:58:55.049580+0000","last_fullsized":"2024-03-24T02:58:55.049580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:07:40.078639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528824+0000","last_change":"2024-03-24T02:58:36.980107+0000","last_active":"2024-03-24T02:58:43.528824+0000","last_peered":"2024-03-24T02:58:43.528824+0000","last_clean":"2024-03-24T02:58:43.528824+0000","last_became_active":"2024-03-24T02:58:36.978914+0000","last_became_peered":"2024-03-24T02:58:36.978914+0000","last_unstale":"2024-03-24T02:58:43.528824+0000","last_undegraded":"2024-03-24T02:58:43.528824+0000","last_fullsized":"2024-03-24T02:58:43.528824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:43:36.956326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986042+0000","last_change":"2024-03-24T02:58:40.996582+0000","last_active":"2024-03-24T02:58:41.986042+0000","last_peered":"2024-03-24T02:58:41.986042+0000","last_clean":"2024-03-24T02:58:41.986042+0000","last_became_active":"2024-03-24T02:58:40.993673+0000","last_became_peered":"2024-03-24T02:58:40.993673+0000","last_unstale":"2024-03-24T02:58:41.986042+0000","last_undegraded":"2024-03-24T02:58:41.986042+0000","last_fullsized":"2024-03-24T02:58:41.986042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:07:34.506740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'11","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.050343+0000","last_change":"2024-03-24T02:58:36.130829+0000","last_active":"2024-03-24T02:58:55.050343+0000","last_peered":"2024-03-24T02:58:55.050343+0000","last_clean":"2024-03-24T02:58:55.050343+0000","last_became_active":"2024-03-24T02:58:36.130483+0000","last_became_peered":"2024-03-24T02:58:36.130483+0000","last_unstale":"2024-03-24T02:58:55.050343+0000","last_undegraded":"2024-03-24T02:58:55.050343+0000","last_fullsized":"2024-03-24T02:58:55.050343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:40:32.407363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986492+0000","last_change":"2024-03-24T02:58:36.984011+0000","last_active":"2024-03-24T02:58:41.986492+0000","last_peered":"2024-03-24T02:58:41.986492+0000","last_clean":"2024-03-24T02:58:41.986492+0000","last_became_active":"2024-03-24T02:58:36.983623+0000","last_became_peered":"2024-03-24T02:58:36.983623+0000","last_unstale":"2024-03-24T02:58:41.986492+0000","last_undegraded":"2024-03-24T02:58:41.986492+0000","last_fullsized":"2024-03-24T02:58:41.986492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:54:35.745458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528809+0000","last_change":"2024-03-24T02:58:40.993361+0000","last_active":"2024-03-24T02:58:43.528809+0000","last_peered":"2024-03-24T02:58:43.528809+0000","last_clean":"2024-03-24T02:58:43.528809+0000","last_became_active":"2024-03-24T02:58:40.992983+0000","last_became_peered":"2024-03-24T02:58:40.992983+0000","last_unstale":"2024-03-24T02:58:43.528809+0000","last_undegraded":"2024-03-24T02:58:43.528809+0000","last_fullsized":"2024-03-24T02:58:43.528809+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:04:21.700904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:59.092818+0000","last_change":"2024-03-24T02:58:36.131101+0000","last_active":"2024-03-24T02:58:59.092818+0000","last_peered":"2024-03-24T02:58:59.092818+0000","last_clean":"2024-03-24T02:58:59.092818+0000","last_became_active":"2024-03-24T02:58:36.130356+0000","last_became_peered":"2024-03-24T02:58:36.130356+0000","last_unstale":"2024-03-24T02:58:59.092818+0000","last_undegraded":"2024-03-24T02:58:59.092818+0000","last_fullsized":"2024-03-24T02:58:59.092818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:16:40.613073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527723+0000","last_change":"2024-03-24T02:58:36.983955+0000","last_active":"2024-03-24T02:58:43.527723+0000","last_peered":"2024-03-24T02:58:43.527723+0000","last_clean":"2024-03-24T02:58:43.527723+0000","last_became_active":"2024-03-24T02:58:36.983409+0000","last_became_peered":"2024-03-24T02:58:36.983409+0000","last_unstale":"2024-03-24T02:58:43.527723+0000","last_undegraded":"2024-03-24T02:58:43.527723+0000","last_fullsized":"2024-03-24T02:58:43.527723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:24:14.675779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527739+0000","last_change":"2024-03-24T02:58:41.004609+0000","last_active":"2024-03-24T02:58:43.527739+0000","last_peered":"2024-03-24T02:58:43.527739+0000","last_clean":"2024-03-24T02:58:43.527739+0000","last_became_active":"2024-03-24T02:58:41.004199+0000","last_became_peered":"2024-03-24T02:58:41.004199+0000","last_unstale":"2024-03-24T02:58:43.527739+0000","last_undegraded":"2024-03-24T02:58:43.527739+0000","last_fullsized":"2024-03-24T02:58:43.527739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:06:49.717152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:57.066403+0000","last_change":"2024-03-24T02:58:36.137678+0000","last_active":"2024-03-24T02:58:57.066403+0000","last_peered":"2024-03-24T02:58:57.066403+0000","last_clean":"2024-03-24T02:58:57.066403+0000","last_became_active":"2024-03-24T02:58:36.137213+0000","last_became_peered":"2024-03-24T02:58:36.137213+0000","last_unstale":"2024-03-24T02:58:57.066403+0000","last_undegraded":"2024-03-24T02:58:57.066403+0000","last_fullsized":"2024-03-24T02:58:57.066403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:27:50.610128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.989000+0000","last_change":"2024-03-24T02:58:36.981976+0000","last_active":"2024-03-24T02:58:41.989000+0000","last_peered":"2024-03-24T02:58:41.989000+0000","last_clean":"2024-03-24T02:58:41.989000+0000","last_became_active":"2024-03-24T02:58:36.981617+0000","last_became_peered":"2024-03-24T02:58:36.981617+0000","last_unstale":"2024-03-24T02:58:41.989000+0000","last_undegraded":"2024-03-24T02:58:41.989000+0000","last_fullsized":"2024-03-24T02:58:41.989000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:39:43.822859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986368+0000","last_change":"2024-03-24T02:58:41.001497+0000","last_active":"2024-03-24T02:58:41.986368+0000","last_peered":"2024-03-24T02:58:41.986368+0000","last_clean":"2024-03-24T02:58:41.986368+0000","last_became_active":"2024-03-24T02:58:41.001088+0000","last_became_peered":"2024-03-24T02:58:41.001088+0000","last_unstale":"2024-03-24T02:58:41.986368+0000","last_undegraded":"2024-03-24T02:58:41.986368+0000","last_fullsized":"2024-03-24T02:58:41.986368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:26:51.912623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:59.093015+0000","last_change":"2024-03-24T02:58:36.135245+0000","last_active":"2024-03-24T02:58:59.093015+0000","last_peered":"2024-03-24T02:58:59.093015+0000","last_clean":"2024-03-24T02:58:59.093015+0000","last_became_active":"2024-03-24T02:58:36.134853+0000","last_became_peered":"2024-03-24T02:58:36.134853+0000","last_unstale":"2024-03-24T02:58:59.093015+0000","last_undegraded":"2024-03-24T02:58:59.093015+0000","last_fullsized":"2024-03-24T02:58:59.093015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:50:11.890138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.988982+0000","last_change":"2024-03-24T02:58:36.988925+0000","last_active":"2024-03-24T02:58:41.988982+0000","last_peered":"2024-03-24T02:58:41.988982+0000","last_clean":"2024-03-24T02:58:41.988982+0000","last_became_active":"2024-03-24T02:58:36.987201+0000","last_became_peered":"2024-03-24T02:58:36.987201+0000","last_unstale":"2024-03-24T02:58:41.988982+0000","last_undegraded":"2024-03-24T02:58:41.988982+0000","last_fullsized":"2024-03-24T02:58:41.988982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:38:54.277890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528451+0000","last_change":"2024-03-24T02:58:41.002868+0000","last_active":"2024-03-24T02:58:43.528451+0000","last_peered":"2024-03-24T02:58:43.528451+0000","last_clean":"2024-03-24T02:58:43.528451+0000","last_became_active":"2024-03-24T02:58:41.002436+0000","last_became_peered":"2024-03-24T02:58:41.002436+0000","last_unstale":"2024-03-24T02:58:43.528451+0000","last_undegraded":"2024-03-24T02:58:43.528451+0000","last_fullsized":"2024-03-24T02:58:43.528451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:31:41.870769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'10","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.047292+0000","last_change":"2024-03-24T02:58:36.127705+0000","last_active":"2024-03-24T02:58:55.047292+0000","last_peered":"2024-03-24T02:58:55.047292+0000","last_clean":"2024-03-24T02:58:55.047292+0000","last_became_active":"2024-03-24T02:58:36.127359+0000","last_became_peered":"2024-03-24T02:58:36.127359+0000","last_unstale":"2024-03-24T02:58:55.047292+0000","last_undegraded":"2024-03-24T02:58:55.047292+0000","last_fullsized":"2024-03-24T02:58:55.047292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:14:13.504039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528005+0000","last_change":"2024-03-24T02:58:36.986285+0000","last_active":"2024-03-24T02:58:43.528005+0000","last_peered":"2024-03-24T02:58:43.528005+0000","last_clean":"2024-03-24T02:58:43.528005+0000","last_became_active":"2024-03-24T02:58:36.983690+0000","last_became_peered":"2024-03-24T02:58:36.983690+0000","last_unstale":"2024-03-24T02:58:43.528005+0000","last_undegraded":"2024-03-24T02:58:43.528005+0000","last_fullsized":"2024-03-24T02:58:43.528005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:14:13.903312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528148+0000","last_change":"2024-03-24T02:58:40.994348+0000","last_active":"2024-03-24T02:58:43.528148+0000","last_peered":"2024-03-24T02:58:43.528148+0000","last_clean":"2024-03-24T02:58:43.528148+0000","last_became_active":"2024-03-24T02:58:40.993871+0000","last_became_peered":"2024-03-24T02:58:40.993871+0000","last_unstale":"2024-03-24T02:58:43.528148+0000","last_undegraded":"2024-03-24T02:58:43.528148+0000","last_fullsized":"2024-03-24T02:58:43.528148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:29:29.236204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986437+0000","last_change":"2024-03-24T02:58:36.130645+0000","last_active":"2024-03-24T02:58:41.986437+0000","last_peered":"2024-03-24T02:58:41.986437+0000","last_clean":"2024-03-24T02:58:41.986437+0000","last_became_active":"2024-03-24T02:58:36.130289+0000","last_became_peered":"2024-03-24T02:58:36.130289+0000","last_unstale":"2024-03-24T02:58:41.986437+0000","last_undegraded":"2024-03-24T02:58:41.986437+0000","last_fullsized":"2024-03-24T02:58:41.986437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:18:04.085684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527787+0000","last_change":"2024-03-24T02:58:36.980899+0000","last_active":"2024-03-24T02:58:43.527787+0000","last_peered":"2024-03-24T02:58:43.527787+0000","last_clean":"2024-03-24T02:58:43.527787+0000","last_became_active":"2024-03-24T02:58:36.979893+0000","last_became_peered":"2024-03-24T02:58:36.979893+0000","last_unstale":"2024-03-24T02:58:43.527787+0000","last_undegraded":"2024-03-24T02:58:43.527787+0000","last_fullsized":"2024-03-24T02:58:43.527787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:39.208269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986260+0000","last_change":"2024-03-24T02:58:40.986381+0000","last_active":"2024-03-24T02:58:41.986260+0000","last_peered":"2024-03-24T02:58:41.986260+0000","last_clean":"2024-03-24T02:58:41.986260+0000","last_became_active":"2024-03-24T02:58:40.985539+0000","last_became_peered":"2024-03-24T02:58:40.985539+0000","last_unstale":"2024-03-24T02:58:41.986260+0000","last_undegraded":"2024-03-24T02:58:41.986260+0000","last_fullsized":"2024-03-24T02:58:41.986260+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:10:13.593975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986143+0000","last_change":"2024-03-24T02:58:36.126405+0000","last_active":"2024-03-24T02:58:41.986143+0000","last_peered":"2024-03-24T02:58:41.986143+0000","last_clean":"2024-03-24T02:58:41.986143+0000","last_became_active":"2024-03-24T02:58:36.126055+0000","last_became_peered":"2024-03-24T02:58:36.126055+0000","last_unstale":"2024-03-24T02:58:41.986143+0000","last_undegraded":"2024-03-24T02:58:41.986143+0000","last_fullsized":"2024-03-24T02:58:41.986143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:04:36.647320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.990398+0000","last_change":"2024-03-24T02:58:36.967546+0000","last_active":"2024-03-24T02:58:42.990398+0000","last_peered":"2024-03-24T02:58:42.990398+0000","last_clean":"2024-03-24T02:58:42.990398+0000","last_became_active":"2024-03-24T02:58:36.967158+0000","last_became_peered":"2024-03-24T02:58:36.967158+0000","last_unstale":"2024-03-24T02:58:42.990398+0000","last_undegraded":"2024-03-24T02:58:42.990398+0000","last_fullsized":"2024-03-24T02:58:42.990398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:06:43.500531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528181+0000","last_change":"2024-03-24T02:58:41.005489+0000","last_active":"2024-03-24T02:58:43.528181+0000","last_peered":"2024-03-24T02:58:43.528181+0000","last_clean":"2024-03-24T02:58:43.528181+0000","last_became_active":"2024-03-24T02:58:41.004948+0000","last_became_peered":"2024-03-24T02:58:41.004948+0000","last_unstale":"2024-03-24T02:58:43.528181+0000","last_undegraded":"2024-03-24T02:58:43.528181+0000","last_fullsized":"2024-03-24T02:58:43.528181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:19:58.161486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:59.095505+0000","last_change":"2024-03-24T02:58:36.136598+0000","last_active":"2024-03-24T02:58:59.095505+0000","last_peered":"2024-03-24T02:58:59.095505+0000","last_clean":"2024-03-24T02:58:59.095505+0000","last_became_active":"2024-03-24T02:58:36.136210+0000","last_became_peered":"2024-03-24T02:58:36.136210+0000","last_unstale":"2024-03-24T02:58:59.095505+0000","last_undegraded":"2024-03-24T02:58:59.095505+0000","last_fullsized":"2024-03-24T02:58:59.095505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:28:51.216582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.990074+0000","last_change":"2024-03-24T02:58:36.976982+0000","last_active":"2024-03-24T02:58:42.990074+0000","last_peered":"2024-03-24T02:58:42.990074+0000","last_clean":"2024-03-24T02:58:42.990074+0000","last_became_active":"2024-03-24T02:58:36.976429+0000","last_became_peered":"2024-03-24T02:58:36.976429+0000","last_unstale":"2024-03-24T02:58:42.990074+0000","last_undegraded":"2024-03-24T02:58:42.990074+0000","last_fullsized":"2024-03-24T02:58:42.990074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:27:37.153099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528459+0000","last_change":"2024-03-24T02:58:41.003368+0000","last_active":"2024-03-24T02:58:43.528459+0000","last_peered":"2024-03-24T02:58:43.528459+0000","last_clean":"2024-03-24T02:58:43.528459+0000","last_became_active":"2024-03-24T02:58:41.002950+0000","last_became_peered":"2024-03-24T02:58:41.002950+0000","last_unstale":"2024-03-24T02:58:43.528459+0000","last_undegraded":"2024-03-24T02:58:43.528459+0000","last_fullsized":"2024-03-24T02:58:43.528459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:27:52.410491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527820+0000","last_change":"2024-03-24T02:58:36.981969+0000","last_active":"2024-03-24T02:58:43.527820+0000","last_peered":"2024-03-24T02:58:43.527820+0000","last_clean":"2024-03-24T02:58:43.527820+0000","last_became_active":"2024-03-24T02:58:36.981535+0000","last_became_peered":"2024-03-24T02:58:36.981535+0000","last_unstale":"2024-03-24T02:58:43.527820+0000","last_undegraded":"2024-03-24T02:58:43.527820+0000","last_fullsized":"2024-03-24T02:58:43.527820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:45:02.572616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:56.059826+0000","last_change":"2024-03-24T02:58:36.130277+0000","last_active":"2024-03-24T02:58:56.059826+0000","last_peered":"2024-03-24T02:58:56.059826+0000","last_clean":"2024-03-24T02:58:56.059826+0000","last_became_active":"2024-03-24T02:58:36.129909+0000","last_became_peered":"2024-03-24T02:58:36.129909+0000","last_unstale":"2024-03-24T02:58:56.059826+0000","last_undegraded":"2024-03-24T02:58:56.059826+0000","last_fullsized":"2024-03-24T02:58:56.059826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:11:30.314752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:44.369571+0000","last_change":"2024-03-24T02:58:40.991037+0000","last_active":"2024-03-24T02:58:44.369571+0000","last_peered":"2024-03-24T02:58:44.369571+0000","last_clean":"2024-03-24T02:58:44.369571+0000","last_became_active":"2024-03-24T02:58:40.990382+0000","last_became_peered":"2024-03-24T02:58:40.990382+0000","last_unstale":"2024-03-24T02:58:44.369571+0000","last_undegraded":"2024-03-24T02:58:44.369571+0000","last_fullsized":"2024-03-24T02:58:44.369571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:08:00.048972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.026847+0000","last_change":"2024-03-24T02:58:36.976255+0000","last_active":"2024-03-24T02:58:43.026847+0000","last_peered":"2024-03-24T02:58:43.026847+0000","last_clean":"2024-03-24T02:58:43.026847+0000","last_became_active":"2024-03-24T02:58:36.975937+0000","last_became_peered":"2024-03-24T02:58:36.975937+0000","last_unstale":"2024-03-24T02:58:43.026847+0000","last_undegraded":"2024-03-24T02:58:43.026847+0000","last_fullsized":"2024-03-24T02:58:43.026847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:44:11.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:59.090839+0000","last_change":"2024-03-24T02:58:36.126466+0000","last_active":"2024-03-24T02:58:59.090839+0000","last_peered":"2024-03-24T02:58:59.090839+0000","last_clean":"2024-03-24T02:58:59.090839+0000","last_became_active":"2024-03-24T02:58:36.125944+0000","last_became_peered":"2024-03-24T02:58:36.125944+0000","last_unstale":"2024-03-24T02:58:59.090839+0000","last_undegraded":"2024-03-24T02:58:59.090839+0000","last_fullsized":"2024-03-24T02:58:59.090839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:37:17.123640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.988625+0000","last_change":"2024-03-24T02:57:20.795020+0000","last_active":"2024-03-24T02:58:41.988625+0000","last_peered":"2024-03-24T02:58:41.988625+0000","last_clean":"2024-03-24T02:58:41.988625+0000","last_became_active":"2024-03-24T02:57:20.794535+0000","last_became_peered":"2024-03-24T02:57:20.794535+0000","last_unstale":"2024-03-24T02:58:41.988625+0000","last_undegraded":"2024-03-24T02:58:41.988625+0000","last_fullsized":"2024-03-24T02:58:41.988625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:53:54.827889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:53:54.827889+0000","last_clean_scrub_stamp":"2024-03-24T02:53:54.827889+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:47:16.501404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528749+0000","last_change":"2024-03-24T02:58:41.002766+0000","last_active":"2024-03-24T02:58:43.528749+0000","last_peered":"2024-03-24T02:58:43.528749+0000","last_clean":"2024-03-24T02:58:43.528749+0000","last_became_active":"2024-03-24T02:58:41.002401+0000","last_became_peered":"2024-03-24T02:58:41.002401+0000","last_unstale":"2024-03-24T02:58:43.528749+0000","last_undegraded":"2024-03-24T02:58:43.528749+0000","last_fullsized":"2024-03-24T02:58:43.528749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:58:31.401629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527636+0000","last_change":"2024-03-24T02:58:36.978649+0000","last_active":"2024-03-24T02:58:43.527636+0000","last_peered":"2024-03-24T02:58:43.527636+0000","last_clean":"2024-03-24T02:58:43.527636+0000","last_became_active":"2024-03-24T02:58:36.978136+0000","last_became_peered":"2024-03-24T02:58:36.978136+0000","last_unstale":"2024-03-24T02:58:43.527636+0000","last_undegraded":"2024-03-24T02:58:43.527636+0000","last_fullsized":"2024-03-24T02:58:43.527636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:43:55.598250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:56.056706+0000","last_change":"2024-03-24T02:58:36.138886+0000","last_active":"2024-03-24T02:58:56.056706+0000","last_peered":"2024-03-24T02:58:56.056706+0000","last_clean":"2024-03-24T02:58:56.056706+0000","last_became_active":"2024-03-24T02:58:36.138582+0000","last_became_peered":"2024-03-24T02:58:36.138582+0000","last_unstale":"2024-03-24T02:58:56.056706+0000","last_undegraded":"2024-03-24T02:58:56.056706+0000","last_fullsized":"2024-03-24T02:58:56.056706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:17:48.129273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527701+0000","last_change":"2024-03-24T02:58:41.000477+0000","last_active":"2024-03-24T02:58:43.527701+0000","last_peered":"2024-03-24T02:58:43.527701+0000","last_clean":"2024-03-24T02:58:43.527701+0000","last_became_active":"2024-03-24T02:58:41.000157+0000","last_became_peered":"2024-03-24T02:58:41.000157+0000","last_unstale":"2024-03-24T02:58:43.527701+0000","last_undegraded":"2024-03-24T02:58:43.527701+0000","last_fullsized":"2024-03-24T02:58:43.527701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:55:44.870347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986599+0000","last_change":"2024-03-24T02:58:36.135524+0000","last_active":"2024-03-24T02:58:41.986599+0000","last_peered":"2024-03-24T02:58:41.986599+0000","last_clean":"2024-03-24T02:58:41.986599+0000","last_became_active":"2024-03-24T02:58:36.135170+0000","last_became_peered":"2024-03-24T02:58:36.135170+0000","last_unstale":"2024-03-24T02:58:41.986599+0000","last_undegraded":"2024-03-24T02:58:41.986599+0000","last_fullsized":"2024-03-24T02:58:41.986599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:56:52.737193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528249+0000","last_change":"2024-03-24T02:58:36.969887+0000","last_active":"2024-03-24T02:58:43.528249+0000","last_peered":"2024-03-24T02:58:43.528249+0000","last_clean":"2024-03-24T02:58:43.528249+0000","last_became_active":"2024-03-24T02:58:36.969510+0000","last_became_peered":"2024-03-24T02:58:36.969510+0000","last_unstale":"2024-03-24T02:58:43.528249+0000","last_undegraded":"2024-03-24T02:58:43.528249+0000","last_fullsized":"2024-03-24T02:58:43.528249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:55:58.135452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.026760+0000","last_change":"2024-03-24T02:58:40.996393+0000","last_active":"2024-03-24T02:58:43.026760+0000","last_peered":"2024-03-24T02:58:43.026760+0000","last_clean":"2024-03-24T02:58:43.026760+0000","last_became_active":"2024-03-24T02:58:40.995967+0000","last_became_peered":"2024-03-24T02:58:40.995967+0000","last_unstale":"2024-03-24T02:58:43.026760+0000","last_undegraded":"2024-03-24T02:58:43.026760+0000","last_fullsized":"2024-03-24T02:58:43.026760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:14:51.160730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:56.058239+0000","last_change":"2024-03-24T02:58:36.135033+0000","last_active":"2024-03-24T02:58:56.058239+0000","last_peered":"2024-03-24T02:58:56.058239+0000","last_clean":"2024-03-24T02:58:56.058239+0000","last_became_active":"2024-03-24T02:58:36.134713+0000","last_became_peered":"2024-03-24T02:58:36.134713+0000","last_unstale":"2024-03-24T02:58:56.058239+0000","last_undegraded":"2024-03-24T02:58:56.058239+0000","last_fullsized":"2024-03-24T02:58:56.058239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:05:55.777016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.027332+0000","last_change":"2024-03-24T02:58:36.984024+0000","last_active":"2024-03-24T02:58:43.027332+0000","last_peered":"2024-03-24T02:58:43.027332+0000","last_clean":"2024-03-24T02:58:43.027332+0000","last_became_active":"2024-03-24T02:58:36.983659+0000","last_became_peered":"2024-03-24T02:58:36.983659+0000","last_unstale":"2024-03-24T02:58:43.027332+0000","last_undegraded":"2024-03-24T02:58:43.027332+0000","last_fullsized":"2024-03-24T02:58:43.027332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:47:59.900749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575406+0000","last_change":"2024-03-24T02:58:40.991181+0000","last_active":"2024-03-24T02:58:42.575406+0000","last_peered":"2024-03-24T02:58:42.575406+0000","last_clean":"2024-03-24T02:58:42.575406+0000","last_became_active":"2024-03-24T02:58:40.990755+0000","last_became_peered":"2024-03-24T02:58:40.990755+0000","last_unstale":"2024-03-24T02:58:42.575406+0000","last_undegraded":"2024-03-24T02:58:42.575406+0000","last_fullsized":"2024-03-24T02:58:42.575406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:57:05.480364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:58.083378+0000","last_change":"2024-03-24T02:58:36.123100+0000","last_active":"2024-03-24T02:58:58.083378+0000","last_peered":"2024-03-24T02:58:58.083378+0000","last_clean":"2024-03-24T02:58:58.083378+0000","last_became_active":"2024-03-24T02:58:36.122613+0000","last_became_peered":"2024-03-24T02:58:36.122613+0000","last_unstale":"2024-03-24T02:58:58.083378+0000","last_undegraded":"2024-03-24T02:58:58.083378+0000","last_fullsized":"2024-03-24T02:58:58.083378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:31:23.954532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528065+0000","last_change":"2024-03-24T02:58:36.977010+0000","last_active":"2024-03-24T02:58:43.528065+0000","last_peered":"2024-03-24T02:58:43.528065+0000","last_clean":"2024-03-24T02:58:43.528065+0000","last_became_active":"2024-03-24T02:58:36.976499+0000","last_became_peered":"2024-03-24T02:58:36.976499+0000","last_unstale":"2024-03-24T02:58:43.528065+0000","last_undegraded":"2024-03-24T02:58:43.528065+0000","last_fullsized":"2024-03-24T02:58:43.528065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:18:33.754474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528841+0000","last_change":"2024-03-24T02:58:40.991043+0000","last_active":"2024-03-24T02:58:43.528841+0000","last_peered":"2024-03-24T02:58:43.528841+0000","last_clean":"2024-03-24T02:58:43.528841+0000","last_became_active":"2024-03-24T02:58:40.990549+0000","last_became_peered":"2024-03-24T02:58:40.990549+0000","last_unstale":"2024-03-24T02:58:43.528841+0000","last_undegraded":"2024-03-24T02:58:43.528841+0000","last_fullsized":"2024-03-24T02:58:43.528841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:20:34.014398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.051343+0000","last_change":"2024-03-24T02:58:36.127420+0000","last_active":"2024-03-24T02:58:55.051343+0000","last_peered":"2024-03-24T02:58:55.051343+0000","last_clean":"2024-03-24T02:58:55.051343+0000","last_became_active":"2024-03-24T02:58:36.126995+0000","last_became_peered":"2024-03-24T02:58:36.126995+0000","last_unstale":"2024-03-24T02:58:55.051343+0000","last_undegraded":"2024-03-24T02:58:55.051343+0000","last_fullsized":"2024-03-24T02:58:55.051343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:01:52.250569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986120+0000","last_change":"2024-03-24T02:58:36.971978+0000","last_active":"2024-03-24T02:58:41.986120+0000","last_peered":"2024-03-24T02:58:41.986120+0000","last_clean":"2024-03-24T02:58:41.986120+0000","last_became_active":"2024-03-24T02:58:36.969812+0000","last_became_peered":"2024-03-24T02:58:36.969812+0000","last_unstale":"2024-03-24T02:58:41.986120+0000","last_undegraded":"2024-03-24T02:58:41.986120+0000","last_fullsized":"2024-03-24T02:58:41.986120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:44:38.573916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986527+0000","last_change":"2024-03-24T02:58:40.994515+0000","last_active":"2024-03-24T02:58:41.986527+0000","last_peered":"2024-03-24T02:58:41.986527+0000","last_clean":"2024-03-24T02:58:41.986527+0000","last_became_active":"2024-03-24T02:58:40.994113+0000","last_became_peered":"2024-03-24T02:58:40.994113+0000","last_unstale":"2024-03-24T02:58:41.986527+0000","last_undegraded":"2024-03-24T02:58:41.986527+0000","last_fullsized":"2024-03-24T02:58:41.986527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:21:58.972660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:57.066384+0000","last_change":"2024-03-24T02:58:36.136590+0000","last_active":"2024-03-24T02:58:57.066384+0000","last_peered":"2024-03-24T02:58:57.066384+0000","last_clean":"2024-03-24T02:58:57.066384+0000","last_became_active":"2024-03-24T02:58:36.136248+0000","last_became_peered":"2024-03-24T02:58:36.136248+0000","last_unstale":"2024-03-24T02:58:57.066384+0000","last_undegraded":"2024-03-24T02:58:57.066384+0000","last_fullsized":"2024-03-24T02:58:57.066384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:20:50.738185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527379+0000","last_change":"2024-03-24T02:58:36.977145+0000","last_active":"2024-03-24T02:58:43.527379+0000","last_peered":"2024-03-24T02:58:43.527379+0000","last_clean":"2024-03-24T02:58:43.527379+0000","last_became_active":"2024-03-24T02:58:36.975835+0000","last_became_peered":"2024-03-24T02:58:36.975835+0000","last_unstale":"2024-03-24T02:58:43.527379+0000","last_undegraded":"2024-03-24T02:58:43.527379+0000","last_fullsized":"2024-03-24T02:58:43.527379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:33:30.150675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:44.293204+0000","last_change":"2024-03-24T02:58:41.005505+0000","last_active":"2024-03-24T02:58:44.293204+0000","last_peered":"2024-03-24T02:58:44.293204+0000","last_clean":"2024-03-24T02:58:44.293204+0000","last_became_active":"2024-03-24T02:58:41.005063+0000","last_became_peered":"2024-03-24T02:58:41.005063+0000","last_unstale":"2024-03-24T02:58:44.293204+0000","last_undegraded":"2024-03-24T02:58:44.293204+0000","last_fullsized":"2024-03-24T02:58:44.293204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:22:31.947209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:57.066136+0000","last_change":"2024-03-24T02:58:36.130936+0000","last_active":"2024-03-24T02:58:57.066136+0000","last_peered":"2024-03-24T02:58:57.066136+0000","last_clean":"2024-03-24T02:58:57.066136+0000","last_became_active":"2024-03-24T02:58:36.130581+0000","last_became_peered":"2024-03-24T02:58:36.130581+0000","last_unstale":"2024-03-24T02:58:57.066136+0000","last_undegraded":"2024-03-24T02:58:57.066136+0000","last_fullsized":"2024-03-24T02:58:57.066136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:52.958392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.027156+0000","last_change":"2024-03-24T02:58:36.977062+0000","last_active":"2024-03-24T02:58:43.027156+0000","last_peered":"2024-03-24T02:58:43.027156+0000","last_clean":"2024-03-24T02:58:43.027156+0000","last_became_active":"2024-03-24T02:58:36.976675+0000","last_became_peered":"2024-03-24T02:58:36.976675+0000","last_unstale":"2024-03-24T02:58:43.027156+0000","last_undegraded":"2024-03-24T02:58:43.027156+0000","last_fullsized":"2024-03-24T02:58:43.027156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:37:31.955379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528832+0000","last_change":"2024-03-24T02:58:40.996227+0000","last_active":"2024-03-24T02:58:43.528832+0000","last_peered":"2024-03-24T02:58:43.528832+0000","last_clean":"2024-03-24T02:58:43.528832+0000","last_became_active":"2024-03-24T02:58:40.995394+0000","last_became_peered":"2024-03-24T02:58:40.995394+0000","last_unstale":"2024-03-24T02:58:43.528832+0000","last_undegraded":"2024-03-24T02:58:43.528832+0000","last_fullsized":"2024-03-24T02:58:43.528832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:00:04.370712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.050513+0000","last_change":"2024-03-24T02:58:36.135777+0000","last_active":"2024-03-24T02:58:55.050513+0000","last_peered":"2024-03-24T02:58:55.050513+0000","last_clean":"2024-03-24T02:58:55.050513+0000","last_became_active":"2024-03-24T02:58:36.135398+0000","last_became_peered":"2024-03-24T02:58:36.135398+0000","last_unstale":"2024-03-24T02:58:55.050513+0000","last_undegraded":"2024-03-24T02:58:55.050513+0000","last_fullsized":"2024-03-24T02:58:55.050513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:11:01.641323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.989415+0000","last_change":"2024-03-24T02:58:36.981004+0000","last_active":"2024-03-24T02:58:41.989415+0000","last_peered":"2024-03-24T02:58:41.989415+0000","last_clean":"2024-03-24T02:58:41.989415+0000","last_became_active":"2024-03-24T02:58:36.980258+0000","last_became_peered":"2024-03-24T02:58:36.980258+0000","last_unstale":"2024-03-24T02:58:41.989415+0000","last_undegraded":"2024-03-24T02:58:41.989415+0000","last_fullsized":"2024-03-24T02:58:41.989415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:21:22.479122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528206+0000","last_change":"2024-03-24T02:58:40.994636+0000","last_active":"2024-03-24T02:58:43.528206+0000","last_peered":"2024-03-24T02:58:43.528206+0000","last_clean":"2024-03-24T02:58:43.528206+0000","last_became_active":"2024-03-24T02:58:40.994122+0000","last_became_peered":"2024-03-24T02:58:40.994122+0000","last_unstale":"2024-03-24T02:58:43.528206+0000","last_undegraded":"2024-03-24T02:58:43.528206+0000","last_fullsized":"2024-03-24T02:58:43.528206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:55:56.707430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:59.095540+0000","last_change":"2024-03-24T02:58:36.134528+0000","last_active":"2024-03-24T02:58:59.095540+0000","last_peered":"2024-03-24T02:58:59.095540+0000","last_clean":"2024-03-24T02:58:59.095540+0000","last_became_active":"2024-03-24T02:58:36.134158+0000","last_became_peered":"2024-03-24T02:58:36.134158+0000","last_unstale":"2024-03-24T02:58:59.095540+0000","last_undegraded":"2024-03-24T02:58:59.095540+0000","last_fullsized":"2024-03-24T02:58:59.095540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:50:31.582575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.989380+0000","last_change":"2024-03-24T02:58:36.984545+0000","last_active":"2024-03-24T02:58:41.989380+0000","last_peered":"2024-03-24T02:58:41.989380+0000","last_clean":"2024-03-24T02:58:41.989380+0000","last_became_active":"2024-03-24T02:58:36.982843+0000","last_became_peered":"2024-03-24T02:58:36.982843+0000","last_unstale":"2024-03-24T02:58:41.989380+0000","last_undegraded":"2024-03-24T02:58:41.989380+0000","last_fullsized":"2024-03-24T02:58:41.989380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:50:04.481097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527531+0000","last_change":"2024-03-24T02:58:41.000190+0000","last_active":"2024-03-24T02:58:43.527531+0000","last_peered":"2024-03-24T02:58:43.527531+0000","last_clean":"2024-03-24T02:58:43.527531+0000","last_became_active":"2024-03-24T02:58:40.999767+0000","last_became_peered":"2024-03-24T02:58:40.999767+0000","last_unstale":"2024-03-24T02:58:43.527531+0000","last_undegraded":"2024-03-24T02:58:43.527531+0000","last_fullsized":"2024-03-24T02:58:43.527531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:31:44.927277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:57.066482+0000","last_change":"2024-03-24T02:58:36.133118+0000","last_active":"2024-03-24T02:58:57.066482+0000","last_peered":"2024-03-24T02:58:57.066482+0000","last_clean":"2024-03-24T02:58:57.066482+0000","last_became_active":"2024-03-24T02:58:36.132747+0000","last_became_peered":"2024-03-24T02:58:36.132747+0000","last_unstale":"2024-03-24T02:58:57.066482+0000","last_undegraded":"2024-03-24T02:58:57.066482+0000","last_fullsized":"2024-03-24T02:58:57.066482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:11:15.665601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.988973+0000","last_change":"2024-03-24T02:58:36.985489+0000","last_active":"2024-03-24T02:58:41.988973+0000","last_peered":"2024-03-24T02:58:41.988973+0000","last_clean":"2024-03-24T02:58:41.988973+0000","last_became_active":"2024-03-24T02:58:36.984421+0000","last_became_peered":"2024-03-24T02:58:36.984421+0000","last_unstale":"2024-03-24T02:58:41.988973+0000","last_undegraded":"2024-03-24T02:58:41.988973+0000","last_fullsized":"2024-03-24T02:58:41.988973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:57:03.004888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527670+0000","last_change":"2024-03-24T02:58:40.991456+0000","last_active":"2024-03-24T02:58:43.527670+0000","last_peered":"2024-03-24T02:58:43.527670+0000","last_clean":"2024-03-24T02:58:43.527670+0000","last_became_active":"2024-03-24T02:58:40.990973+0000","last_became_peered":"2024-03-24T02:58:40.990973+0000","last_unstale":"2024-03-24T02:58:43.527670+0000","last_undegraded":"2024-03-24T02:58:43.527670+0000","last_fullsized":"2024-03-24T02:58:43.527670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:36:01.276988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:58.081728+0000","last_change":"2024-03-24T02:58:36.130967+0000","last_active":"2024-03-24T02:58:58.081728+0000","last_peered":"2024-03-24T02:58:58.081728+0000","last_clean":"2024-03-24T02:58:58.081728+0000","last_became_active":"2024-03-24T02:58:36.130657+0000","last_became_peered":"2024-03-24T02:58:36.130657+0000","last_unstale":"2024-03-24T02:58:58.081728+0000","last_undegraded":"2024-03-24T02:58:58.081728+0000","last_fullsized":"2024-03-24T02:58:58.081728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:26:20.426922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.989135+0000","last_change":"2024-03-24T02:58:36.982297+0000","last_active":"2024-03-24T02:58:41.989135+0000","last_peered":"2024-03-24T02:58:41.989135+0000","last_clean":"2024-03-24T02:58:41.989135+0000","last_became_active":"2024-03-24T02:58:36.981792+0000","last_became_peered":"2024-03-24T02:58:36.981792+0000","last_unstale":"2024-03-24T02:58:41.989135+0000","last_undegraded":"2024-03-24T02:58:41.989135+0000","last_fullsized":"2024-03-24T02:58:41.989135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:03:07.528142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527432+0000","last_change":"2024-03-24T02:58:36.979815+0000","last_active":"2024-03-24T02:58:43.527432+0000","last_peered":"2024-03-24T02:58:43.527432+0000","last_clean":"2024-03-24T02:58:43.527432+0000","last_became_active":"2024-03-24T02:58:36.979467+0000","last_became_peered":"2024-03-24T02:58:36.979467+0000","last_unstale":"2024-03-24T02:58:43.527432+0000","last_undegraded":"2024-03-24T02:58:43.527432+0000","last_fullsized":"2024-03-24T02:58:43.527432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:28.056630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575292+0000","last_change":"2024-03-24T02:58:36.974644+0000","last_active":"2024-03-24T02:58:42.575292+0000","last_peered":"2024-03-24T02:58:42.575292+0000","last_clean":"2024-03-24T02:58:42.575292+0000","last_became_active":"2024-03-24T02:58:36.974276+0000","last_became_peered":"2024-03-24T02:58:36.974276+0000","last_unstale":"2024-03-24T02:58:42.575292+0000","last_undegraded":"2024-03-24T02:58:42.575292+0000","last_fullsized":"2024-03-24T02:58:42.575292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:04.606926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575217+0000","last_change":"2024-03-24T02:58:36.978207+0000","last_active":"2024-03-24T02:58:42.575217+0000","last_peered":"2024-03-24T02:58:42.575217+0000","last_clean":"2024-03-24T02:58:42.575217+0000","last_became_active":"2024-03-24T02:58:36.977760+0000","last_became_peered":"2024-03-24T02:58:36.977760+0000","last_unstale":"2024-03-24T02:58:42.575217+0000","last_undegraded":"2024-03-24T02:58:42.575217+0000","last_fullsized":"2024-03-24T02:58:42.575217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:43:51.603963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527813+0000","last_change":"2024-03-24T02:58:36.975740+0000","last_active":"2024-03-24T02:58:43.527813+0000","last_peered":"2024-03-24T02:58:43.527813+0000","last_clean":"2024-03-24T02:58:43.527813+0000","last_became_active":"2024-03-24T02:58:36.975316+0000","last_became_peered":"2024-03-24T02:58:36.975316+0000","last_unstale":"2024-03-24T02:58:43.527813+0000","last_undegraded":"2024-03-24T02:58:43.527813+0000","last_fullsized":"2024-03-24T02:58:43.527813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:21:33.579169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528364+0000","last_change":"2024-03-24T02:58:36.977261+0000","last_active":"2024-03-24T02:58:43.528364+0000","last_peered":"2024-03-24T02:58:43.528364+0000","last_clean":"2024-03-24T02:58:43.528364+0000","last_became_active":"2024-03-24T02:58:36.974644+0000","last_became_peered":"2024-03-24T02:58:36.974644+0000","last_unstale":"2024-03-24T02:58:43.528364+0000","last_undegraded":"2024-03-24T02:58:43.528364+0000","last_fullsized":"2024-03-24T02:58:43.528364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:34:50.968770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527857+0000","last_change":"2024-03-24T02:58:36.981223+0000","last_active":"2024-03-24T02:58:43.527857+0000","last_peered":"2024-03-24T02:58:43.527857+0000","last_clean":"2024-03-24T02:58:43.527857+0000","last_became_active":"2024-03-24T02:58:36.979728+0000","last_became_peered":"2024-03-24T02:58:36.979728+0000","last_unstale":"2024-03-24T02:58:43.527857+0000","last_undegraded":"2024-03-24T02:58:43.527857+0000","last_fullsized":"2024-03-24T02:58:43.527857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:08:41.300442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575700+0000","last_change":"2024-03-24T02:58:36.972385+0000","last_active":"2024-03-24T02:58:42.575700+0000","last_peered":"2024-03-24T02:58:42.575700+0000","last_clean":"2024-03-24T02:58:42.575700+0000","last_became_active":"2024-03-24T02:58:36.972018+0000","last_became_peered":"2024-03-24T02:58:36.972018+0000","last_unstale":"2024-03-24T02:58:42.575700+0000","last_undegraded":"2024-03-24T02:58:42.575700+0000","last_fullsized":"2024-03-24T02:58:42.575700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:02:22.529198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.027444+0000","last_change":"2024-03-24T02:58:36.975284+0000","last_active":"2024-03-24T02:58:43.027444+0000","last_peered":"2024-03-24T02:58:43.027444+0000","last_clean":"2024-03-24T02:58:43.027444+0000","last_became_active":"2024-03-24T02:58:36.974942+0000","last_became_peered":"2024-03-24T02:58:36.974942+0000","last_unstale":"2024-03-24T02:58:43.027444+0000","last_undegraded":"2024-03-24T02:58:43.027444+0000","last_fullsized":"2024-03-24T02:58:43.027444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:58:50.093347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527780+0000","last_change":"2024-03-24T02:58:36.977256+0000","last_active":"2024-03-24T02:58:43.527780+0000","last_peered":"2024-03-24T02:58:43.527780+0000","last_clean":"2024-03-24T02:58:43.527780+0000","last_became_active":"2024-03-24T02:58:36.976888+0000","last_became_peered":"2024-03-24T02:58:36.976888+0000","last_unstale":"2024-03-24T02:58:43.527780+0000","last_undegraded":"2024-03-24T02:58:43.527780+0000","last_fullsized":"2024-03-24T02:58:43.527780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:18:19.449590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.026867+0000","last_change":"2024-03-24T02:58:36.981413+0000","last_active":"2024-03-24T02:58:43.026867+0000","last_peered":"2024-03-24T02:58:43.026867+0000","last_clean":"2024-03-24T02:58:43.026867+0000","last_became_active":"2024-03-24T02:58:36.980346+0000","last_became_peered":"2024-03-24T02:58:36.980346+0000","last_unstale":"2024-03-24T02:58:43.026867+0000","last_undegraded":"2024-03-24T02:58:43.026867+0000","last_fullsized":"2024-03-24T02:58:43.026867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:17:57.332228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528552+0000","last_change":"2024-03-24T02:58:36.984264+0000","last_active":"2024-03-24T02:58:43.528552+0000","last_peered":"2024-03-24T02:58:43.528552+0000","last_clean":"2024-03-24T02:58:43.528552+0000","last_became_active":"2024-03-24T02:58:36.983905+0000","last_became_peered":"2024-03-24T02:58:36.983905+0000","last_unstale":"2024-03-24T02:58:43.528552+0000","last_undegraded":"2024-03-24T02:58:43.528552+0000","last_fullsized":"2024-03-24T02:58:43.528552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:25:17.872320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527780+0000","last_change":"2024-03-24T02:58:36.975939+0000","last_active":"2024-03-24T02:58:43.527780+0000","last_peered":"2024-03-24T02:58:43.527780+0000","last_clean":"2024-03-24T02:58:43.527780+0000","last_became_active":"2024-03-24T02:58:36.975477+0000","last_became_peered":"2024-03-24T02:58:36.975477+0000","last_unstale":"2024-03-24T02:58:43.527780+0000","last_undegraded":"2024-03-24T02:58:43.527780+0000","last_fullsized":"2024-03-24T02:58:43.527780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:46:43.603391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527568+0000","last_change":"2024-03-24T02:58:36.980674+0000","last_active":"2024-03-24T02:58:43.527568+0000","last_peered":"2024-03-24T02:58:43.527568+0000","last_clean":"2024-03-24T02:58:43.527568+0000","last_became_active":"2024-03-24T02:58:36.980278+0000","last_became_peered":"2024-03-24T02:58:36.980278+0000","last_unstale":"2024-03-24T02:58:43.527568+0000","last_undegraded":"2024-03-24T02:58:43.527568+0000","last_fullsized":"2024-03-24T02:58:43.527568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:50:01.719362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10172,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":167,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":352256,"data_stored":56574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":478407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1073127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":478033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":477819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1613824,"data_stored":1075212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":477937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":481057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":478693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":511324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":477499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":478306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36920,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708232,"statfs":{"total":95995035648,"available":95957229568,"internally_reserved":0,"allocated":1622016,"data_stored":1070403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":34109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":5544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-24T02:59:00.539 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-24T02:59:00.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:00 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] up:active 2024-03-24T02:59:00.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:00 smithi188 ceph-mon[36206]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-24T02:59:00.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:00 smithi188 ceph-mon[36206]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-24T02:59:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:00 smithi188 ceph-mon[36206]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-24T02:59:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:00 smithi188 ceph-mon[36206]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:59:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:00 smithi188 ceph-mon[36206]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:59:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:00 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3448624485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:59:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:00 smithi188 ceph-mon[36206]: pgmap v304: 129 pgs: 129 active+clean; 587 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-03-24T02:59:00.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:00 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] up:active 2024-03-24T02:59:00.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:00 smithi019 ceph-mon[35944]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-24T02:59:00.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:00 smithi019 ceph-mon[35944]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-24T02:59:00.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:00 smithi019 ceph-mon[35944]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-24T02:59:00.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:00 smithi019 ceph-mon[35944]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:59:00.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:00 smithi019 ceph-mon[35944]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T02:59:00.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:00 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3448624485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T02:59:00.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:00 smithi019 ceph-mon[35944]: pgmap v304: 129 pgs: 129 active+clean; 587 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-03-24T02:59:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.677+0000 7fb58a713640 1 -- 172.21.15.17:0/187961171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5841016c0 msgr2=0x7fb584101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.677+0000 7fb58a713640 1 --2- 172.21.15.17:0/187961171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5841016c0 0x7fb584101ae0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb578009900 tx=0x7fb57802bc30 comp rx=0 tx=0).stop 2024-03-24T02:59:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.677+0000 7fb58a713640 1 -- 172.21.15.17:0/187961171 shutdown_connections 2024-03-24T02:59:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.677+0000 7fb58a713640 1 --2- 172.21.15.17:0/187961171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb584103270 0x7fb584109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.677+0000 7fb58a713640 1 --2- 172.21.15.17:0/187961171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5841028b0 0x7fb584102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.677+0000 7fb58a713640 1 --2- 172.21.15.17:0/187961171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5841016c0 0x7fb584101ae0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.677+0000 7fb58a713640 1 -- 172.21.15.17:0/187961171 >> 172.21.15.17:0/187961171 conn(0x7fb5840fcc50 msgr2=0x7fb5840ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.677+0000 7fb58a713640 1 -- 172.21.15.17:0/187961171 shutdown_connections 2024-03-24T02:59:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.677+0000 7fb58a713640 1 -- 172.21.15.17:0/187961171 wait complete. 2024-03-24T02:59:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb58a713640 1 Processor -- start 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb58a713640 1 -- start start 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb58a713640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5841016c0 0x7fb584197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb58a713640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5841028b0 0x7fb584197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb58a713640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb584103270 0x7fb58419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb58a713640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58410d340 con 0x7fb5841016c0 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb58a713640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58410d1c0 con 0x7fb584103270 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb58a713640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58410d4c0 con 0x7fb5841028b0 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb583fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5841016c0 0x7fb584197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb583fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5841016c0 0x7fb584197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59248/0 (socket says 172.21.15.17:59248) 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb583fff640 1 -- 172.21.15.17:0/3521827663 learned_addr learned my addr 172.21.15.17:0/3521827663 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb588cb0640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb584103270 0x7fb58419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.678+0000 7fb5837fe640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5841028b0 0x7fb584197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.679+0000 7fb583fff640 1 -- 172.21.15.17:0/3521827663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5841028b0 msgr2=0x7fb584197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.679+0000 7fb583fff640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5841028b0 0x7fb584197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.679+0000 7fb583fff640 1 -- 172.21.15.17:0/3521827663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb584103270 msgr2=0x7fb58419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.679+0000 7fb583fff640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb584103270 0x7fb58419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.679+0000 7fb583fff640 1 -- 172.21.15.17:0/3521827663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58419f630 con 0x7fb5841016c0 2024-03-24T02:59:00.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.679+0000 7fb583fff640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5841016c0 0x7fb584197a10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb578000c00 tx=0x7fb57802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:00.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.679+0000 7fb5817fa640 1 -- 172.21.15.17:0/3521827663 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578041070 con 0x7fb5841016c0 2024-03-24T02:59:00.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.679+0000 7fb5817fa640 1 -- 172.21.15.17:0/3521827663 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb578004450 con 0x7fb5841016c0 2024-03-24T02:59:00.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.679+0000 7fb58a713640 1 -- 172.21.15.17:0/3521827663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58419f8c0 con 0x7fb5841016c0 2024-03-24T02:59:00.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.679+0000 7fb5817fa640 1 -- 172.21.15.17:0/3521827663 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578034440 con 0x7fb5841016c0 2024-03-24T02:59:00.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.679+0000 7fb58a713640 1 -- 172.21.15.17:0/3521827663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58419fdb0 con 0x7fb5841016c0 2024-03-24T02:59:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.680+0000 7fb58a713640 1 -- 172.21.15.17:0/3521827663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb584108a60 con 0x7fb5841016c0 2024-03-24T02:59:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.681+0000 7fb5817fa640 1 -- 172.21.15.17:0/3521827663 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fb578044070 con 0x7fb5841016c0 2024-03-24T02:59:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.682+0000 7fb5817fa640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb56c0a2890 0x7fb56c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.682+0000 7fb5837fe640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb56c0a2890 0x7fb56c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.682+0000 7fb5817fa640 1 -- 172.21.15.17:0/3521827663 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb5780f07e0 con 0x7fb5841016c0 2024-03-24T02:59:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.682+0000 7fb5837fe640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb56c0a2890 0x7fb56c0a4d50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb584198f20 tx=0x7fb574005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:00.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.686+0000 7fb5817fa640 1 -- 172.21.15.17:0/3521827663 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb5780bd030 con 0x7fb5841016c0 2024-03-24T02:59:00.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.867+0000 7fb58a713640 1 -- 172.21.15.17:0/3521827663 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb584061ae0 con 0x7fb56c0a2890 2024-03-24T02:59:00.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.877+0000 7fb5817fa640 1 -- 172.21.15.17:0/3521827663 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340401 (secure 0 0 0) 0x7fb584061ae0 con 0x7fb56c0a2890 2024-03-24T02:59:00.879 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:59:00.881 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-24T02:59:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.880+0000 7fb58a713640 1 -- 172.21.15.17:0/3521827663 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb56c0a2890 msgr2=0x7fb56c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.880+0000 7fb58a713640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb56c0a2890 0x7fb56c0a4d50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb584198f20 tx=0x7fb574005f90 comp rx=0 tx=0).stop 2024-03-24T02:59:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.880+0000 7fb58a713640 1 -- 172.21.15.17:0/3521827663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5841016c0 msgr2=0x7fb584197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.880+0000 7fb58a713640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5841016c0 0x7fb584197a10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb578000c00 tx=0x7fb57802c640 comp rx=0 tx=0).stop 2024-03-24T02:59:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.880+0000 7fb58a713640 1 -- 172.21.15.17:0/3521827663 shutdown_connections 2024-03-24T02:59:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.880+0000 7fb58a713640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb56c0a2890 0x7fb56c0a4d50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.880+0000 7fb58a713640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb584103270 0x7fb58419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.880+0000 7fb58a713640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5841028b0 0x7fb584197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.880+0000 7fb58a713640 1 --2- 172.21.15.17:0/3521827663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5841016c0 0x7fb584197a10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.880+0000 7fb58a713640 1 -- 172.21.15.17:0/3521827663 >> 172.21.15.17:0/3521827663 conn(0x7fb5840fcc50 msgr2=0x7fb584105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.881+0000 7fb58a713640 1 -- 172.21.15.17:0/3521827663 shutdown_connections 2024-03-24T02:59:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:00.881+0000 7fb58a713640 1 -- 172.21.15.17:0/3521827663 wait complete. 2024-03-24T02:59:00.890 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-03-24T02:58:59.950482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600774,"num_objects":82,"num_object_clones":0,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":152,"num_read_kb":132,"num_write":350,"num_write_kb":2997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":286,"ondisk_log_size":286,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404356,"kb_used_data":13596,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537468,"statfs":{"total":1151940427776,"available":1151526367232,"internally_reserved":0,"allocated":13922304,"data_stored":7557817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7996,"num_objects":58,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":124,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003650"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527985+0000","last_change":"2024-03-24T02:58:36.978486+0000","last_active":"2024-03-24T02:58:43.527985+0000","last_peered":"2024-03-24T02:58:43.527985+0000","last_clean":"2024-03-24T02:58:43.527985+0000","last_became_active":"2024-03-24T02:58:36.976937+0000","last_became_peered":"2024-03-24T02:58:36.976937+0000","last_unstale":"2024-03-24T02:58:43.527985+0000","last_undegraded":"2024-03-24T02:58:43.527985+0000","last_fullsized":"2024-03-24T02:58:43.527985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:25:09.439179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528138+0000","last_change":"2024-03-24T02:58:36.983050+0000","last_active":"2024-03-24T02:58:43.528138+0000","last_peered":"2024-03-24T02:58:43.528138+0000","last_clean":"2024-03-24T02:58:43.528138+0000","last_became_active":"2024-03-24T02:58:36.982741+0000","last_became_peered":"2024-03-24T02:58:36.982741+0000","last_unstale":"2024-03-24T02:58:43.528138+0000","last_undegraded":"2024-03-24T02:58:43.528138+0000","last_fullsized":"2024-03-24T02:58:43.528138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:26:35.938559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527776+0000","last_change":"2024-03-24T02:58:36.976313+0000","last_active":"2024-03-24T02:58:43.527776+0000","last_peered":"2024-03-24T02:58:43.527776+0000","last_clean":"2024-03-24T02:58:43.527776+0000","last_became_active":"2024-03-24T02:58:36.975087+0000","last_became_peered":"2024-03-24T02:58:36.975087+0000","last_unstale":"2024-03-24T02:58:43.527776+0000","last_undegraded":"2024-03-24T02:58:43.527776+0000","last_fullsized":"2024-03-24T02:58:43.527776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:22:31.476824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528524+0000","last_change":"2024-03-24T02:58:36.977632+0000","last_active":"2024-03-24T02:58:43.528524+0000","last_peered":"2024-03-24T02:58:43.528524+0000","last_clean":"2024-03-24T02:58:43.528524+0000","last_became_active":"2024-03-24T02:58:36.977111+0000","last_became_peered":"2024-03-24T02:58:36.977111+0000","last_unstale":"2024-03-24T02:58:43.528524+0000","last_undegraded":"2024-03-24T02:58:43.528524+0000","last_fullsized":"2024-03-24T02:58:43.528524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:47:07.874538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986247+0000","last_change":"2024-03-24T02:58:36.967543+0000","last_active":"2024-03-24T02:58:41.986247+0000","last_peered":"2024-03-24T02:58:41.986247+0000","last_clean":"2024-03-24T02:58:41.986247+0000","last_became_active":"2024-03-24T02:58:36.967108+0000","last_became_peered":"2024-03-24T02:58:36.967108+0000","last_unstale":"2024-03-24T02:58:41.986247+0000","last_undegraded":"2024-03-24T02:58:41.986247+0000","last_fullsized":"2024-03-24T02:58:41.986247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:38:46.280891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.989906+0000","last_change":"2024-03-24T02:58:36.977777+0000","last_active":"2024-03-24T02:58:42.989906+0000","last_peered":"2024-03-24T02:58:42.989906+0000","last_clean":"2024-03-24T02:58:42.989906+0000","last_became_active":"2024-03-24T02:58:36.976615+0000","last_became_peered":"2024-03-24T02:58:36.976615+0000","last_unstale":"2024-03-24T02:58:42.989906+0000","last_undegraded":"2024-03-24T02:58:42.989906+0000","last_fullsized":"2024-03-24T02:58:42.989906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:13:15.149963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.989245+0000","last_change":"2024-03-24T02:58:36.990832+0000","last_active":"2024-03-24T02:58:41.989245+0000","last_peered":"2024-03-24T02:58:41.989245+0000","last_clean":"2024-03-24T02:58:41.989245+0000","last_became_active":"2024-03-24T02:58:36.990395+0000","last_became_peered":"2024-03-24T02:58:36.990395+0000","last_unstale":"2024-03-24T02:58:41.989245+0000","last_undegraded":"2024-03-24T02:58:41.989245+0000","last_fullsized":"2024-03-24T02:58:41.989245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:34.915192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.026791+0000","last_change":"2024-03-24T02:58:36.980829+0000","last_active":"2024-03-24T02:58:43.026791+0000","last_peered":"2024-03-24T02:58:43.026791+0000","last_clean":"2024-03-24T02:58:43.026791+0000","last_became_active":"2024-03-24T02:58:36.978641+0000","last_became_peered":"2024-03-24T02:58:36.978641+0000","last_unstale":"2024-03-24T02:58:43.026791+0000","last_undegraded":"2024-03-24T02:58:43.026791+0000","last_fullsized":"2024-03-24T02:58:43.026791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:56:29.948080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.529167+0000","last_change":"2024-03-24T02:58:36.989173+0000","last_active":"2024-03-24T02:58:43.529167+0000","last_peered":"2024-03-24T02:58:43.529167+0000","last_clean":"2024-03-24T02:58:43.529167+0000","last_became_active":"2024-03-24T02:58:36.988771+0000","last_became_peered":"2024-03-24T02:58:36.988771+0000","last_unstale":"2024-03-24T02:58:43.529167+0000","last_undegraded":"2024-03-24T02:58:43.529167+0000","last_fullsized":"2024-03-24T02:58:43.529167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:51:01.790952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.990653+0000","last_change":"2024-03-24T02:58:36.978505+0000","last_active":"2024-03-24T02:58:42.990653+0000","last_peered":"2024-03-24T02:58:42.990653+0000","last_clean":"2024-03-24T02:58:42.990653+0000","last_became_active":"2024-03-24T02:58:36.978145+0000","last_became_peered":"2024-03-24T02:58:36.978145+0000","last_unstale":"2024-03-24T02:58:42.990653+0000","last_undegraded":"2024-03-24T02:58:42.990653+0000","last_fullsized":"2024-03-24T02:58:42.990653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:06:41.874675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.989375+0000","last_change":"2024-03-24T02:58:36.984134+0000","last_active":"2024-03-24T02:58:41.989375+0000","last_peered":"2024-03-24T02:58:41.989375+0000","last_clean":"2024-03-24T02:58:41.989375+0000","last_became_active":"2024-03-24T02:58:36.981037+0000","last_became_peered":"2024-03-24T02:58:36.981037+0000","last_unstale":"2024-03-24T02:58:41.989375+0000","last_undegraded":"2024-03-24T02:58:41.989375+0000","last_fullsized":"2024-03-24T02:58:41.989375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:06:40.023840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528397+0000","last_change":"2024-03-24T02:58:36.979852+0000","last_active":"2024-03-24T02:58:43.528397+0000","last_peered":"2024-03-24T02:58:43.528397+0000","last_clean":"2024-03-24T02:58:43.528397+0000","last_became_active":"2024-03-24T02:58:36.979506+0000","last_became_peered":"2024-03-24T02:58:36.979506+0000","last_unstale":"2024-03-24T02:58:43.528397+0000","last_undegraded":"2024-03-24T02:58:43.528397+0000","last_fullsized":"2024-03-24T02:58:43.528397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:01:27.493694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527939+0000","last_change":"2024-03-24T02:58:36.977018+0000","last_active":"2024-03-24T02:58:43.527939+0000","last_peered":"2024-03-24T02:58:43.527939+0000","last_clean":"2024-03-24T02:58:43.527939+0000","last_became_active":"2024-03-24T02:58:36.976669+0000","last_became_peered":"2024-03-24T02:58:36.976669+0000","last_unstale":"2024-03-24T02:58:43.527939+0000","last_undegraded":"2024-03-24T02:58:43.527939+0000","last_fullsized":"2024-03-24T02:58:43.527939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:23:15.049093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527483+0000","last_change":"2024-03-24T02:58:36.973200+0000","last_active":"2024-03-24T02:58:43.527483+0000","last_peered":"2024-03-24T02:58:43.527483+0000","last_clean":"2024-03-24T02:58:43.527483+0000","last_became_active":"2024-03-24T02:58:36.972828+0000","last_became_peered":"2024-03-24T02:58:36.972828+0000","last_unstale":"2024-03-24T02:58:43.527483+0000","last_undegraded":"2024-03-24T02:58:43.527483+0000","last_fullsized":"2024-03-24T02:58:43.527483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:42:25.122881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527580+0000","last_change":"2024-03-24T02:58:36.977686+0000","last_active":"2024-03-24T02:58:43.527580+0000","last_peered":"2024-03-24T02:58:43.527580+0000","last_clean":"2024-03-24T02:58:43.527580+0000","last_became_active":"2024-03-24T02:58:36.976311+0000","last_became_peered":"2024-03-24T02:58:36.976311+0000","last_unstale":"2024-03-24T02:58:43.527580+0000","last_undegraded":"2024-03-24T02:58:43.527580+0000","last_fullsized":"2024-03-24T02:58:43.527580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:10:04.585726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528141+0000","last_change":"2024-03-24T02:58:36.985336+0000","last_active":"2024-03-24T02:58:43.528141+0000","last_peered":"2024-03-24T02:58:43.528141+0000","last_clean":"2024-03-24T02:58:43.528141+0000","last_became_active":"2024-03-24T02:58:36.984842+0000","last_became_peered":"2024-03-24T02:58:36.984842+0000","last_unstale":"2024-03-24T02:58:43.528141+0000","last_undegraded":"2024-03-24T02:58:43.528141+0000","last_fullsized":"2024-03-24T02:58:43.528141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:52:41.819132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527858+0000","last_change":"2024-03-24T02:58:36.978310+0000","last_active":"2024-03-24T02:58:43.527858+0000","last_peered":"2024-03-24T02:58:43.527858+0000","last_clean":"2024-03-24T02:58:43.527858+0000","last_became_active":"2024-03-24T02:58:36.977396+0000","last_became_peered":"2024-03-24T02:58:36.977396+0000","last_unstale":"2024-03-24T02:58:43.527858+0000","last_undegraded":"2024-03-24T02:58:43.527858+0000","last_fullsized":"2024-03-24T02:58:43.527858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:37:21.200591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986366+0000","last_change":"2024-03-24T02:58:36.979514+0000","last_active":"2024-03-24T02:58:41.986366+0000","last_peered":"2024-03-24T02:58:41.986366+0000","last_clean":"2024-03-24T02:58:41.986366+0000","last_became_active":"2024-03-24T02:58:36.979155+0000","last_became_peered":"2024-03-24T02:58:36.979155+0000","last_unstale":"2024-03-24T02:58:41.986366+0000","last_undegraded":"2024-03-24T02:58:41.986366+0000","last_fullsized":"2024-03-24T02:58:41.986366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:49:37.142195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575668+0000","last_change":"2024-03-24T02:58:36.971546+0000","last_active":"2024-03-24T02:58:42.575668+0000","last_peered":"2024-03-24T02:58:42.575668+0000","last_clean":"2024-03-24T02:58:42.575668+0000","last_became_active":"2024-03-24T02:58:36.971210+0000","last_became_peered":"2024-03-24T02:58:36.971210+0000","last_unstale":"2024-03-24T02:58:42.575668+0000","last_undegraded":"2024-03-24T02:58:42.575668+0000","last_fullsized":"2024-03-24T02:58:42.575668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:35:21.274898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575518+0000","last_change":"2024-03-24T02:58:40.994331+0000","last_active":"2024-03-24T02:58:42.575518+0000","last_peered":"2024-03-24T02:58:42.575518+0000","last_clean":"2024-03-24T02:58:42.575518+0000","last_became_active":"2024-03-24T02:58:40.993880+0000","last_became_peered":"2024-03-24T02:58:40.993880+0000","last_unstale":"2024-03-24T02:58:42.575518+0000","last_undegraded":"2024-03-24T02:58:42.575518+0000","last_fullsized":"2024-03-24T02:58:42.575518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:58:46.876872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:56.059840+0000","last_change":"2024-03-24T02:58:36.132975+0000","last_active":"2024-03-24T02:58:56.059840+0000","last_peered":"2024-03-24T02:58:56.059840+0000","last_clean":"2024-03-24T02:58:56.059840+0000","last_became_active":"2024-03-24T02:58:36.132640+0000","last_became_peered":"2024-03-24T02:58:36.132640+0000","last_unstale":"2024-03-24T02:58:56.059840+0000","last_undegraded":"2024-03-24T02:58:56.059840+0000","last_fullsized":"2024-03-24T02:58:56.059840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:37:03.765773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.027263+0000","last_change":"2024-03-24T02:58:36.977283+0000","last_active":"2024-03-24T02:58:43.027263+0000","last_peered":"2024-03-24T02:58:43.027263+0000","last_clean":"2024-03-24T02:58:43.027263+0000","last_became_active":"2024-03-24T02:58:36.976968+0000","last_became_peered":"2024-03-24T02:58:36.976968+0000","last_unstale":"2024-03-24T02:58:43.027263+0000","last_undegraded":"2024-03-24T02:58:43.027263+0000","last_fullsized":"2024-03-24T02:58:43.027263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:28:12.115521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527852+0000","last_change":"2024-03-24T02:58:40.997896+0000","last_active":"2024-03-24T02:58:43.527852+0000","last_peered":"2024-03-24T02:58:43.527852+0000","last_clean":"2024-03-24T02:58:43.527852+0000","last_became_active":"2024-03-24T02:58:40.996845+0000","last_became_peered":"2024-03-24T02:58:40.996845+0000","last_unstale":"2024-03-24T02:58:43.527852+0000","last_undegraded":"2024-03-24T02:58:43.527852+0000","last_fullsized":"2024-03-24T02:58:43.527852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:51:17.031332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.027447+0000","last_change":"2024-03-24T02:58:36.139069+0000","last_active":"2024-03-24T02:58:43.027447+0000","last_peered":"2024-03-24T02:58:43.027447+0000","last_clean":"2024-03-24T02:58:43.027447+0000","last_became_active":"2024-03-24T02:58:36.138571+0000","last_became_peered":"2024-03-24T02:58:36.138571+0000","last_unstale":"2024-03-24T02:58:43.027447+0000","last_undegraded":"2024-03-24T02:58:43.027447+0000","last_fullsized":"2024-03-24T02:58:43.027447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:17:44.510904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528182+0000","last_change":"2024-03-24T02:58:36.976727+0000","last_active":"2024-03-24T02:58:43.528182+0000","last_peered":"2024-03-24T02:58:43.528182+0000","last_clean":"2024-03-24T02:58:43.528182+0000","last_became_active":"2024-03-24T02:58:36.976076+0000","last_became_peered":"2024-03-24T02:58:36.976076+0000","last_unstale":"2024-03-24T02:58:43.528182+0000","last_undegraded":"2024-03-24T02:58:43.528182+0000","last_fullsized":"2024-03-24T02:58:43.528182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:33:03.490882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528014+0000","last_change":"2024-03-24T02:58:41.003025+0000","last_active":"2024-03-24T02:58:43.528014+0000","last_peered":"2024-03-24T02:58:43.528014+0000","last_clean":"2024-03-24T02:58:43.528014+0000","last_became_active":"2024-03-24T02:58:41.002270+0000","last_became_peered":"2024-03-24T02:58:41.002270+0000","last_unstale":"2024-03-24T02:58:43.528014+0000","last_undegraded":"2024-03-24T02:58:43.528014+0000","last_fullsized":"2024-03-24T02:58:43.528014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:31:36.390191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986643+0000","last_change":"2024-03-24T02:58:36.135836+0000","last_active":"2024-03-24T02:58:41.986643+0000","last_peered":"2024-03-24T02:58:41.986643+0000","last_clean":"2024-03-24T02:58:41.986643+0000","last_became_active":"2024-03-24T02:58:36.135476+0000","last_became_peered":"2024-03-24T02:58:36.135476+0000","last_unstale":"2024-03-24T02:58:41.986643+0000","last_undegraded":"2024-03-24T02:58:41.986643+0000","last_fullsized":"2024-03-24T02:58:41.986643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:30:14.058373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528539+0000","last_change":"2024-03-24T02:58:36.981600+0000","last_active":"2024-03-24T02:58:43.528539+0000","last_peered":"2024-03-24T02:58:43.528539+0000","last_clean":"2024-03-24T02:58:43.528539+0000","last_became_active":"2024-03-24T02:58:36.981222+0000","last_became_peered":"2024-03-24T02:58:36.981222+0000","last_unstale":"2024-03-24T02:58:43.528539+0000","last_undegraded":"2024-03-24T02:58:43.528539+0000","last_fullsized":"2024-03-24T02:58:43.528539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:11:33.771157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528011+0000","last_change":"2024-03-24T02:58:40.996838+0000","last_active":"2024-03-24T02:58:43.528011+0000","last_peered":"2024-03-24T02:58:43.528011+0000","last_clean":"2024-03-24T02:58:43.528011+0000","last_became_active":"2024-03-24T02:58:40.996320+0000","last_became_peered":"2024-03-24T02:58:40.996320+0000","last_unstale":"2024-03-24T02:58:43.528011+0000","last_undegraded":"2024-03-24T02:58:43.528011+0000","last_fullsized":"2024-03-24T02:58:43.528011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:11:34.124402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986402+0000","last_change":"2024-03-24T02:58:36.132406+0000","last_active":"2024-03-24T02:58:41.986402+0000","last_peered":"2024-03-24T02:58:41.986402+0000","last_clean":"2024-03-24T02:58:41.986402+0000","last_became_active":"2024-03-24T02:58:36.132044+0000","last_became_peered":"2024-03-24T02:58:36.132044+0000","last_unstale":"2024-03-24T02:58:41.986402+0000","last_undegraded":"2024-03-24T02:58:41.986402+0000","last_fullsized":"2024-03-24T02:58:41.986402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:35:53.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528508+0000","last_change":"2024-03-24T02:58:36.977989+0000","last_active":"2024-03-24T02:58:43.528508+0000","last_peered":"2024-03-24T02:58:43.528508+0000","last_clean":"2024-03-24T02:58:43.528508+0000","last_became_active":"2024-03-24T02:58:36.977653+0000","last_became_peered":"2024-03-24T02:58:36.977653+0000","last_unstale":"2024-03-24T02:58:43.528508+0000","last_undegraded":"2024-03-24T02:58:43.528508+0000","last_fullsized":"2024-03-24T02:58:43.528508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:21:18.364057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575573+0000","last_change":"2024-03-24T02:58:40.989082+0000","last_active":"2024-03-24T02:58:42.575573+0000","last_peered":"2024-03-24T02:58:42.575573+0000","last_clean":"2024-03-24T02:58:42.575573+0000","last_became_active":"2024-03-24T02:58:40.988731+0000","last_became_peered":"2024-03-24T02:58:40.988731+0000","last_unstale":"2024-03-24T02:58:42.575573+0000","last_undegraded":"2024-03-24T02:58:42.575573+0000","last_fullsized":"2024-03-24T02:58:42.575573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:34:39.489446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.050467+0000","last_change":"2024-03-24T02:58:36.135785+0000","last_active":"2024-03-24T02:58:55.050467+0000","last_peered":"2024-03-24T02:58:55.050467+0000","last_clean":"2024-03-24T02:58:55.050467+0000","last_became_active":"2024-03-24T02:58:36.135470+0000","last_became_peered":"2024-03-24T02:58:36.135470+0000","last_unstale":"2024-03-24T02:58:55.050467+0000","last_undegraded":"2024-03-24T02:58:55.050467+0000","last_fullsized":"2024-03-24T02:58:55.050467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:41:26.529773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986204+0000","last_change":"2024-03-24T02:58:36.978241+0000","last_active":"2024-03-24T02:58:41.986204+0000","last_peered":"2024-03-24T02:58:41.986204+0000","last_clean":"2024-03-24T02:58:41.986204+0000","last_became_active":"2024-03-24T02:58:36.977879+0000","last_became_peered":"2024-03-24T02:58:36.977879+0000","last_unstale":"2024-03-24T02:58:41.986204+0000","last_undegraded":"2024-03-24T02:58:41.986204+0000","last_fullsized":"2024-03-24T02:58:41.986204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:44:41.190736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575612+0000","last_change":"2024-03-24T02:58:41.004890+0000","last_active":"2024-03-24T02:58:42.575612+0000","last_peered":"2024-03-24T02:58:42.575612+0000","last_clean":"2024-03-24T02:58:42.575612+0000","last_became_active":"2024-03-24T02:58:41.004428+0000","last_became_peered":"2024-03-24T02:58:41.004428+0000","last_unstale":"2024-03-24T02:58:42.575612+0000","last_undegraded":"2024-03-24T02:58:42.575612+0000","last_fullsized":"2024-03-24T02:58:42.575612+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:43:29.488561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:57.066003+0000","last_change":"2024-03-24T02:58:36.132242+0000","last_active":"2024-03-24T02:58:57.066003+0000","last_peered":"2024-03-24T02:58:57.066003+0000","last_clean":"2024-03-24T02:58:57.066003+0000","last_became_active":"2024-03-24T02:58:36.131792+0000","last_became_peered":"2024-03-24T02:58:36.131792+0000","last_unstale":"2024-03-24T02:58:57.066003+0000","last_undegraded":"2024-03-24T02:58:57.066003+0000","last_fullsized":"2024-03-24T02:58:57.066003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:09:16.751023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528421+0000","last_change":"2024-03-24T02:58:36.977306+0000","last_active":"2024-03-24T02:58:43.528421+0000","last_peered":"2024-03-24T02:58:43.528421+0000","last_clean":"2024-03-24T02:58:43.528421+0000","last_became_active":"2024-03-24T02:58:36.976955+0000","last_became_peered":"2024-03-24T02:58:36.976955+0000","last_unstale":"2024-03-24T02:58:43.528421+0000","last_undegraded":"2024-03-24T02:58:43.528421+0000","last_fullsized":"2024-03-24T02:58:43.528421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:09:56.568781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527915+0000","last_change":"2024-03-24T02:58:40.999977+0000","last_active":"2024-03-24T02:58:43.527915+0000","last_peered":"2024-03-24T02:58:43.527915+0000","last_clean":"2024-03-24T02:58:43.527915+0000","last_became_active":"2024-03-24T02:58:40.999564+0000","last_became_peered":"2024-03-24T02:58:40.999564+0000","last_unstale":"2024-03-24T02:58:43.527915+0000","last_undegraded":"2024-03-24T02:58:43.527915+0000","last_fullsized":"2024-03-24T02:58:43.527915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:16:58.022961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:56.058808+0000","last_change":"2024-03-24T02:58:36.123622+0000","last_active":"2024-03-24T02:58:56.058808+0000","last_peered":"2024-03-24T02:58:56.058808+0000","last_clean":"2024-03-24T02:58:56.058808+0000","last_became_active":"2024-03-24T02:58:36.123221+0000","last_became_peered":"2024-03-24T02:58:36.123221+0000","last_unstale":"2024-03-24T02:58:56.058808+0000","last_undegraded":"2024-03-24T02:58:56.058808+0000","last_fullsized":"2024-03-24T02:58:56.058808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:05:26.781268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.026678+0000","last_change":"2024-03-24T02:58:36.983446+0000","last_active":"2024-03-24T02:58:43.026678+0000","last_peered":"2024-03-24T02:58:43.026678+0000","last_clean":"2024-03-24T02:58:43.026678+0000","last_became_active":"2024-03-24T02:58:36.981401+0000","last_became_peered":"2024-03-24T02:58:36.981401+0000","last_unstale":"2024-03-24T02:58:43.026678+0000","last_undegraded":"2024-03-24T02:58:43.026678+0000","last_fullsized":"2024-03-24T02:58:43.026678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:12:42.836678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528020+0000","last_change":"2024-03-24T02:58:40.999348+0000","last_active":"2024-03-24T02:58:43.528020+0000","last_peered":"2024-03-24T02:58:43.528020+0000","last_clean":"2024-03-24T02:58:43.528020+0000","last_became_active":"2024-03-24T02:58:40.998932+0000","last_became_peered":"2024-03-24T02:58:40.998932+0000","last_unstale":"2024-03-24T02:58:43.528020+0000","last_undegraded":"2024-03-24T02:58:43.528020+0000","last_fullsized":"2024-03-24T02:58:43.528020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:24:29.294464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:57.064801+0000","last_change":"2024-03-24T02:58:36.137640+0000","last_active":"2024-03-24T02:58:57.064801+0000","last_peered":"2024-03-24T02:58:57.064801+0000","last_clean":"2024-03-24T02:58:57.064801+0000","last_became_active":"2024-03-24T02:58:36.137237+0000","last_became_peered":"2024-03-24T02:58:36.137237+0000","last_unstale":"2024-03-24T02:58:57.064801+0000","last_undegraded":"2024-03-24T02:58:57.064801+0000","last_fullsized":"2024-03-24T02:58:57.064801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:52:52.246968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.985962+0000","last_change":"2024-03-24T02:58:36.971696+0000","last_active":"2024-03-24T02:58:41.985962+0000","last_peered":"2024-03-24T02:58:41.985962+0000","last_clean":"2024-03-24T02:58:41.985962+0000","last_became_active":"2024-03-24T02:58:36.971318+0000","last_became_peered":"2024-03-24T02:58:36.971318+0000","last_unstale":"2024-03-24T02:58:41.985962+0000","last_undegraded":"2024-03-24T02:58:41.985962+0000","last_fullsized":"2024-03-24T02:58:41.985962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:01:30.976284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.026757+0000","last_change":"2024-03-24T02:58:40.995887+0000","last_active":"2024-03-24T02:58:43.026757+0000","last_peered":"2024-03-24T02:58:43.026757+0000","last_clean":"2024-03-24T02:58:43.026757+0000","last_became_active":"2024-03-24T02:58:40.995242+0000","last_became_peered":"2024-03-24T02:58:40.995242+0000","last_unstale":"2024-03-24T02:58:43.026757+0000","last_undegraded":"2024-03-24T02:58:43.026757+0000","last_fullsized":"2024-03-24T02:58:43.026757+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:33:40.376374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.050493+0000","last_change":"2024-03-24T02:58:36.128252+0000","last_active":"2024-03-24T02:58:55.050493+0000","last_peered":"2024-03-24T02:58:55.050493+0000","last_clean":"2024-03-24T02:58:55.050493+0000","last_became_active":"2024-03-24T02:58:36.127888+0000","last_became_peered":"2024-03-24T02:58:36.127888+0000","last_unstale":"2024-03-24T02:58:55.050493+0000","last_undegraded":"2024-03-24T02:58:55.050493+0000","last_fullsized":"2024-03-24T02:58:55.050493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:31:39.341123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.529063+0000","last_change":"2024-03-24T02:58:36.983836+0000","last_active":"2024-03-24T02:58:43.529063+0000","last_peered":"2024-03-24T02:58:43.529063+0000","last_clean":"2024-03-24T02:58:43.529063+0000","last_became_active":"2024-03-24T02:58:36.983481+0000","last_became_peered":"2024-03-24T02:58:36.983481+0000","last_unstale":"2024-03-24T02:58:43.529063+0000","last_undegraded":"2024-03-24T02:58:43.529063+0000","last_fullsized":"2024-03-24T02:58:43.529063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:01:01.707796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.985994+0000","last_change":"2024-03-24T02:58:41.000176+0000","last_active":"2024-03-24T02:58:41.985994+0000","last_peered":"2024-03-24T02:58:41.985994+0000","last_clean":"2024-03-24T02:58:41.985994+0000","last_became_active":"2024-03-24T02:58:40.999561+0000","last_became_peered":"2024-03-24T02:58:40.999561+0000","last_unstale":"2024-03-24T02:58:41.985994+0000","last_undegraded":"2024-03-24T02:58:41.985994+0000","last_fullsized":"2024-03-24T02:58:41.985994+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:20:28.678497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:58.083714+0000","last_change":"2024-03-24T02:58:36.131588+0000","last_active":"2024-03-24T02:58:58.083714+0000","last_peered":"2024-03-24T02:58:58.083714+0000","last_clean":"2024-03-24T02:58:58.083714+0000","last_became_active":"2024-03-24T02:58:36.131251+0000","last_became_peered":"2024-03-24T02:58:36.131251+0000","last_unstale":"2024-03-24T02:58:58.083714+0000","last_undegraded":"2024-03-24T02:58:58.083714+0000","last_fullsized":"2024-03-24T02:58:58.083714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:03:22.978352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528073+0000","last_change":"2024-03-24T02:58:36.978566+0000","last_active":"2024-03-24T02:58:43.528073+0000","last_peered":"2024-03-24T02:58:43.528073+0000","last_clean":"2024-03-24T02:58:43.528073+0000","last_became_active":"2024-03-24T02:58:36.977950+0000","last_became_peered":"2024-03-24T02:58:36.977950+0000","last_unstale":"2024-03-24T02:58:43.528073+0000","last_undegraded":"2024-03-24T02:58:43.528073+0000","last_fullsized":"2024-03-24T02:58:43.528073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:06:39.029357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528817+0000","last_change":"2024-03-24T02:58:41.003427+0000","last_active":"2024-03-24T02:58:43.528817+0000","last_peered":"2024-03-24T02:58:43.528817+0000","last_clean":"2024-03-24T02:58:43.528817+0000","last_became_active":"2024-03-24T02:58:41.002249+0000","last_became_peered":"2024-03-24T02:58:41.002249+0000","last_unstale":"2024-03-24T02:58:43.528817+0000","last_undegraded":"2024-03-24T02:58:43.528817+0000","last_fullsized":"2024-03-24T02:58:43.528817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:39:36.587451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.049580+0000","last_change":"2024-03-24T02:58:36.132547+0000","last_active":"2024-03-24T02:58:55.049580+0000","last_peered":"2024-03-24T02:58:55.049580+0000","last_clean":"2024-03-24T02:58:55.049580+0000","last_became_active":"2024-03-24T02:58:36.132144+0000","last_became_peered":"2024-03-24T02:58:36.132144+0000","last_unstale":"2024-03-24T02:58:55.049580+0000","last_undegraded":"2024-03-24T02:58:55.049580+0000","last_fullsized":"2024-03-24T02:58:55.049580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:07:40.078639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528824+0000","last_change":"2024-03-24T02:58:36.980107+0000","last_active":"2024-03-24T02:58:43.528824+0000","last_peered":"2024-03-24T02:58:43.528824+0000","last_clean":"2024-03-24T02:58:43.528824+0000","last_became_active":"2024-03-24T02:58:36.978914+0000","last_became_peered":"2024-03-24T02:58:36.978914+0000","last_unstale":"2024-03-24T02:58:43.528824+0000","last_undegraded":"2024-03-24T02:58:43.528824+0000","last_fullsized":"2024-03-24T02:58:43.528824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:43:36.956326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986042+0000","last_change":"2024-03-24T02:58:40.996582+0000","last_active":"2024-03-24T02:58:41.986042+0000","last_peered":"2024-03-24T02:58:41.986042+0000","last_clean":"2024-03-24T02:58:41.986042+0000","last_became_active":"2024-03-24T02:58:40.993673+0000","last_became_peered":"2024-03-24T02:58:40.993673+0000","last_unstale":"2024-03-24T02:58:41.986042+0000","last_undegraded":"2024-03-24T02:58:41.986042+0000","last_fullsized":"2024-03-24T02:58:41.986042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:07:34.506740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'11","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.050343+0000","last_change":"2024-03-24T02:58:36.130829+0000","last_active":"2024-03-24T02:58:55.050343+0000","last_peered":"2024-03-24T02:58:55.050343+0000","last_clean":"2024-03-24T02:58:55.050343+0000","last_became_active":"2024-03-24T02:58:36.130483+0000","last_became_peered":"2024-03-24T02:58:36.130483+0000","last_unstale":"2024-03-24T02:58:55.050343+0000","last_undegraded":"2024-03-24T02:58:55.050343+0000","last_fullsized":"2024-03-24T02:58:55.050343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:40:32.407363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986492+0000","last_change":"2024-03-24T02:58:36.984011+0000","last_active":"2024-03-24T02:58:41.986492+0000","last_peered":"2024-03-24T02:58:41.986492+0000","last_clean":"2024-03-24T02:58:41.986492+0000","last_became_active":"2024-03-24T02:58:36.983623+0000","last_became_peered":"2024-03-24T02:58:36.983623+0000","last_unstale":"2024-03-24T02:58:41.986492+0000","last_undegraded":"2024-03-24T02:58:41.986492+0000","last_fullsized":"2024-03-24T02:58:41.986492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:54:35.745458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528809+0000","last_change":"2024-03-24T02:58:40.993361+0000","last_active":"2024-03-24T02:58:43.528809+0000","last_peered":"2024-03-24T02:58:43.528809+0000","last_clean":"2024-03-24T02:58:43.528809+0000","last_became_active":"2024-03-24T02:58:40.992983+0000","last_became_peered":"2024-03-24T02:58:40.992983+0000","last_unstale":"2024-03-24T02:58:43.528809+0000","last_undegraded":"2024-03-24T02:58:43.528809+0000","last_fullsized":"2024-03-24T02:58:43.528809+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:04:21.700904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:59.092818+0000","last_change":"2024-03-24T02:58:36.131101+0000","last_active":"2024-03-24T02:58:59.092818+0000","last_peered":"2024-03-24T02:58:59.092818+0000","last_clean":"2024-03-24T02:58:59.092818+0000","last_became_active":"2024-03-24T02:58:36.130356+0000","last_became_peered":"2024-03-24T02:58:36.130356+0000","last_unstale":"2024-03-24T02:58:59.092818+0000","last_undegraded":"2024-03-24T02:58:59.092818+0000","last_fullsized":"2024-03-24T02:58:59.092818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:16:40.613073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527723+0000","last_change":"2024-03-24T02:58:36.983955+0000","last_active":"2024-03-24T02:58:43.527723+0000","last_peered":"2024-03-24T02:58:43.527723+0000","last_clean":"2024-03-24T02:58:43.527723+0000","last_became_active":"2024-03-24T02:58:36.983409+0000","last_became_peered":"2024-03-24T02:58:36.983409+0000","last_unstale":"2024-03-24T02:58:43.527723+0000","last_undegraded":"2024-03-24T02:58:43.527723+0000","last_fullsized":"2024-03-24T02:58:43.527723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:24:14.675779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527739+0000","last_change":"2024-03-24T02:58:41.004609+0000","last_active":"2024-03-24T02:58:43.527739+0000","last_peered":"2024-03-24T02:58:43.527739+0000","last_clean":"2024-03-24T02:58:43.527739+0000","last_became_active":"2024-03-24T02:58:41.004199+0000","last_became_peered":"2024-03-24T02:58:41.004199+0000","last_unstale":"2024-03-24T02:58:43.527739+0000","last_undegraded":"2024-03-24T02:58:43.527739+0000","last_fullsized":"2024-03-24T02:58:43.527739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:06:49.717152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:57.066403+0000","last_change":"2024-03-24T02:58:36.137678+0000","last_active":"2024-03-24T02:58:57.066403+0000","last_peered":"2024-03-24T02:58:57.066403+0000","last_clean":"2024-03-24T02:58:57.066403+0000","last_became_active":"2024-03-24T02:58:36.137213+0000","last_became_peered":"2024-03-24T02:58:36.137213+0000","last_unstale":"2024-03-24T02:58:57.066403+0000","last_undegraded":"2024-03-24T02:58:57.066403+0000","last_fullsized":"2024-03-24T02:58:57.066403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:27:50.610128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.989000+0000","last_change":"2024-03-24T02:58:36.981976+0000","last_active":"2024-03-24T02:58:41.989000+0000","last_peered":"2024-03-24T02:58:41.989000+0000","last_clean":"2024-03-24T02:58:41.989000+0000","last_became_active":"2024-03-24T02:58:36.981617+0000","last_became_peered":"2024-03-24T02:58:36.981617+0000","last_unstale":"2024-03-24T02:58:41.989000+0000","last_undegraded":"2024-03-24T02:58:41.989000+0000","last_fullsized":"2024-03-24T02:58:41.989000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:39:43.822859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986368+0000","last_change":"2024-03-24T02:58:41.001497+0000","last_active":"2024-03-24T02:58:41.986368+0000","last_peered":"2024-03-24T02:58:41.986368+0000","last_clean":"2024-03-24T02:58:41.986368+0000","last_became_active":"2024-03-24T02:58:41.001088+0000","last_became_peered":"2024-03-24T02:58:41.001088+0000","last_unstale":"2024-03-24T02:58:41.986368+0000","last_undegraded":"2024-03-24T02:58:41.986368+0000","last_fullsized":"2024-03-24T02:58:41.986368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:26:51.912623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:59.093015+0000","last_change":"2024-03-24T02:58:36.135245+0000","last_active":"2024-03-24T02:58:59.093015+0000","last_peered":"2024-03-24T02:58:59.093015+0000","last_clean":"2024-03-24T02:58:59.093015+0000","last_became_active":"2024-03-24T02:58:36.134853+0000","last_became_peered":"2024-03-24T02:58:36.134853+0000","last_unstale":"2024-03-24T02:58:59.093015+0000","last_undegraded":"2024-03-24T02:58:59.093015+0000","last_fullsized":"2024-03-24T02:58:59.093015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:50:11.890138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.988982+0000","last_change":"2024-03-24T02:58:36.988925+0000","last_active":"2024-03-24T02:58:41.988982+0000","last_peered":"2024-03-24T02:58:41.988982+0000","last_clean":"2024-03-24T02:58:41.988982+0000","last_became_active":"2024-03-24T02:58:36.987201+0000","last_became_peered":"2024-03-24T02:58:36.987201+0000","last_unstale":"2024-03-24T02:58:41.988982+0000","last_undegraded":"2024-03-24T02:58:41.988982+0000","last_fullsized":"2024-03-24T02:58:41.988982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:38:54.277890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528451+0000","last_change":"2024-03-24T02:58:41.002868+0000","last_active":"2024-03-24T02:58:43.528451+0000","last_peered":"2024-03-24T02:58:43.528451+0000","last_clean":"2024-03-24T02:58:43.528451+0000","last_became_active":"2024-03-24T02:58:41.002436+0000","last_became_peered":"2024-03-24T02:58:41.002436+0000","last_unstale":"2024-03-24T02:58:43.528451+0000","last_undegraded":"2024-03-24T02:58:43.528451+0000","last_fullsized":"2024-03-24T02:58:43.528451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:31:41.870769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'10","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.047292+0000","last_change":"2024-03-24T02:58:36.127705+0000","last_active":"2024-03-24T02:58:55.047292+0000","last_peered":"2024-03-24T02:58:55.047292+0000","last_clean":"2024-03-24T02:58:55.047292+0000","last_became_active":"2024-03-24T02:58:36.127359+0000","last_became_peered":"2024-03-24T02:58:36.127359+0000","last_unstale":"2024-03-24T02:58:55.047292+0000","last_undegraded":"2024-03-24T02:58:55.047292+0000","last_fullsized":"2024-03-24T02:58:55.047292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:14:13.504039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528005+0000","last_change":"2024-03-24T02:58:36.986285+0000","last_active":"2024-03-24T02:58:43.528005+0000","last_peered":"2024-03-24T02:58:43.528005+0000","last_clean":"2024-03-24T02:58:43.528005+0000","last_became_active":"2024-03-24T02:58:36.983690+0000","last_became_peered":"2024-03-24T02:58:36.983690+0000","last_unstale":"2024-03-24T02:58:43.528005+0000","last_undegraded":"2024-03-24T02:58:43.528005+0000","last_fullsized":"2024-03-24T02:58:43.528005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:14:13.903312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528148+0000","last_change":"2024-03-24T02:58:40.994348+0000","last_active":"2024-03-24T02:58:43.528148+0000","last_peered":"2024-03-24T02:58:43.528148+0000","last_clean":"2024-03-24T02:58:43.528148+0000","last_became_active":"2024-03-24T02:58:40.993871+0000","last_became_peered":"2024-03-24T02:58:40.993871+0000","last_unstale":"2024-03-24T02:58:43.528148+0000","last_undegraded":"2024-03-24T02:58:43.528148+0000","last_fullsized":"2024-03-24T02:58:43.528148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:29:29.236204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986437+0000","last_change":"2024-03-24T02:58:36.130645+0000","last_active":"2024-03-24T02:58:41.986437+0000","last_peered":"2024-03-24T02:58:41.986437+0000","last_clean":"2024-03-24T02:58:41.986437+0000","last_became_active":"2024-03-24T02:58:36.130289+0000","last_became_peered":"2024-03-24T02:58:36.130289+0000","last_unstale":"2024-03-24T02:58:41.986437+0000","last_undegraded":"2024-03-24T02:58:41.986437+0000","last_fullsized":"2024-03-24T02:58:41.986437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:18:04.085684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527787+0000","last_change":"2024-03-24T02:58:36.980899+0000","last_active":"2024-03-24T02:58:43.527787+0000","last_peered":"2024-03-24T02:58:43.527787+0000","last_clean":"2024-03-24T02:58:43.527787+0000","last_became_active":"2024-03-24T02:58:36.979893+0000","last_became_peered":"2024-03-24T02:58:36.979893+0000","last_unstale":"2024-03-24T02:58:43.527787+0000","last_undegraded":"2024-03-24T02:58:43.527787+0000","last_fullsized":"2024-03-24T02:58:43.527787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:39.208269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986260+0000","last_change":"2024-03-24T02:58:40.986381+0000","last_active":"2024-03-24T02:58:41.986260+0000","last_peered":"2024-03-24T02:58:41.986260+0000","last_clean":"2024-03-24T02:58:41.986260+0000","last_became_active":"2024-03-24T02:58:40.985539+0000","last_became_peered":"2024-03-24T02:58:40.985539+0000","last_unstale":"2024-03-24T02:58:41.986260+0000","last_undegraded":"2024-03-24T02:58:41.986260+0000","last_fullsized":"2024-03-24T02:58:41.986260+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:10:13.593975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986143+0000","last_change":"2024-03-24T02:58:36.126405+0000","last_active":"2024-03-24T02:58:41.986143+0000","last_peered":"2024-03-24T02:58:41.986143+0000","last_clean":"2024-03-24T02:58:41.986143+0000","last_became_active":"2024-03-24T02:58:36.126055+0000","last_became_peered":"2024-03-24T02:58:36.126055+0000","last_unstale":"2024-03-24T02:58:41.986143+0000","last_undegraded":"2024-03-24T02:58:41.986143+0000","last_fullsized":"2024-03-24T02:58:41.986143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:04:36.647320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.990398+0000","last_change":"2024-03-24T02:58:36.967546+0000","last_active":"2024-03-24T02:58:42.990398+0000","last_peered":"2024-03-24T02:58:42.990398+0000","last_clean":"2024-03-24T02:58:42.990398+0000","last_became_active":"2024-03-24T02:58:36.967158+0000","last_became_peered":"2024-03-24T02:58:36.967158+0000","last_unstale":"2024-03-24T02:58:42.990398+0000","last_undegraded":"2024-03-24T02:58:42.990398+0000","last_fullsized":"2024-03-24T02:58:42.990398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:06:43.500531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528181+0000","last_change":"2024-03-24T02:58:41.005489+0000","last_active":"2024-03-24T02:58:43.528181+0000","last_peered":"2024-03-24T02:58:43.528181+0000","last_clean":"2024-03-24T02:58:43.528181+0000","last_became_active":"2024-03-24T02:58:41.004948+0000","last_became_peered":"2024-03-24T02:58:41.004948+0000","last_unstale":"2024-03-24T02:58:43.528181+0000","last_undegraded":"2024-03-24T02:58:43.528181+0000","last_fullsized":"2024-03-24T02:58:43.528181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:19:58.161486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:59.095505+0000","last_change":"2024-03-24T02:58:36.136598+0000","last_active":"2024-03-24T02:58:59.095505+0000","last_peered":"2024-03-24T02:58:59.095505+0000","last_clean":"2024-03-24T02:58:59.095505+0000","last_became_active":"2024-03-24T02:58:36.136210+0000","last_became_peered":"2024-03-24T02:58:36.136210+0000","last_unstale":"2024-03-24T02:58:59.095505+0000","last_undegraded":"2024-03-24T02:58:59.095505+0000","last_fullsized":"2024-03-24T02:58:59.095505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:28:51.216582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.990074+0000","last_change":"2024-03-24T02:58:36.976982+0000","last_active":"2024-03-24T02:58:42.990074+0000","last_peered":"2024-03-24T02:58:42.990074+0000","last_clean":"2024-03-24T02:58:42.990074+0000","last_became_active":"2024-03-24T02:58:36.976429+0000","last_became_peered":"2024-03-24T02:58:36.976429+0000","last_unstale":"2024-03-24T02:58:42.990074+0000","last_undegraded":"2024-03-24T02:58:42.990074+0000","last_fullsized":"2024-03-24T02:58:42.990074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:27:37.153099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528459+0000","last_change":"2024-03-24T02:58:41.003368+0000","last_active":"2024-03-24T02:58:43.528459+0000","last_peered":"2024-03-24T02:58:43.528459+0000","last_clean":"2024-03-24T02:58:43.528459+0000","last_became_active":"2024-03-24T02:58:41.002950+0000","last_became_peered":"2024-03-24T02:58:41.002950+0000","last_unstale":"2024-03-24T02:58:43.528459+0000","last_undegraded":"2024-03-24T02:58:43.528459+0000","last_fullsized":"2024-03-24T02:58:43.528459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:27:52.410491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527820+0000","last_change":"2024-03-24T02:58:36.981969+0000","last_active":"2024-03-24T02:58:43.527820+0000","last_peered":"2024-03-24T02:58:43.527820+0000","last_clean":"2024-03-24T02:58:43.527820+0000","last_became_active":"2024-03-24T02:58:36.981535+0000","last_became_peered":"2024-03-24T02:58:36.981535+0000","last_unstale":"2024-03-24T02:58:43.527820+0000","last_undegraded":"2024-03-24T02:58:43.527820+0000","last_fullsized":"2024-03-24T02:58:43.527820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:45:02.572616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:56.059826+0000","last_change":"2024-03-24T02:58:36.130277+0000","last_active":"2024-03-24T02:58:56.059826+0000","last_peered":"2024-03-24T02:58:56.059826+0000","last_clean":"2024-03-24T02:58:56.059826+0000","last_became_active":"2024-03-24T02:58:36.129909+0000","last_became_peered":"2024-03-24T02:58:36.129909+0000","last_unstale":"2024-03-24T02:58:56.059826+0000","last_undegraded":"2024-03-24T02:58:56.059826+0000","last_fullsized":"2024-03-24T02:58:56.059826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:11:30.314752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:44.369571+0000","last_change":"2024-03-24T02:58:40.991037+0000","last_active":"2024-03-24T02:58:44.369571+0000","last_peered":"2024-03-24T02:58:44.369571+0000","last_clean":"2024-03-24T02:58:44.369571+0000","last_became_active":"2024-03-24T02:58:40.990382+0000","last_became_peered":"2024-03-24T02:58:40.990382+0000","last_unstale":"2024-03-24T02:58:44.369571+0000","last_undegraded":"2024-03-24T02:58:44.369571+0000","last_fullsized":"2024-03-24T02:58:44.369571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:08:00.048972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.026847+0000","last_change":"2024-03-24T02:58:36.976255+0000","last_active":"2024-03-24T02:58:43.026847+0000","last_peered":"2024-03-24T02:58:43.026847+0000","last_clean":"2024-03-24T02:58:43.026847+0000","last_became_active":"2024-03-24T02:58:36.975937+0000","last_became_peered":"2024-03-24T02:58:36.975937+0000","last_unstale":"2024-03-24T02:58:43.026847+0000","last_undegraded":"2024-03-24T02:58:43.026847+0000","last_fullsized":"2024-03-24T02:58:43.026847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:44:11.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:59.090839+0000","last_change":"2024-03-24T02:58:36.126466+0000","last_active":"2024-03-24T02:58:59.090839+0000","last_peered":"2024-03-24T02:58:59.090839+0000","last_clean":"2024-03-24T02:58:59.090839+0000","last_became_active":"2024-03-24T02:58:36.125944+0000","last_became_peered":"2024-03-24T02:58:36.125944+0000","last_unstale":"2024-03-24T02:58:59.090839+0000","last_undegraded":"2024-03-24T02:58:59.090839+0000","last_fullsized":"2024-03-24T02:58:59.090839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:37:17.123640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.988625+0000","last_change":"2024-03-24T02:57:20.795020+0000","last_active":"2024-03-24T02:58:41.988625+0000","last_peered":"2024-03-24T02:58:41.988625+0000","last_clean":"2024-03-24T02:58:41.988625+0000","last_became_active":"2024-03-24T02:57:20.794535+0000","last_became_peered":"2024-03-24T02:57:20.794535+0000","last_unstale":"2024-03-24T02:58:41.988625+0000","last_undegraded":"2024-03-24T02:58:41.988625+0000","last_fullsized":"2024-03-24T02:58:41.988625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:53:54.827889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:53:54.827889+0000","last_clean_scrub_stamp":"2024-03-24T02:53:54.827889+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:47:16.501404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528749+0000","last_change":"2024-03-24T02:58:41.002766+0000","last_active":"2024-03-24T02:58:43.528749+0000","last_peered":"2024-03-24T02:58:43.528749+0000","last_clean":"2024-03-24T02:58:43.528749+0000","last_became_active":"2024-03-24T02:58:41.002401+0000","last_became_peered":"2024-03-24T02:58:41.002401+0000","last_unstale":"2024-03-24T02:58:43.528749+0000","last_undegraded":"2024-03-24T02:58:43.528749+0000","last_fullsized":"2024-03-24T02:58:43.528749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:58:31.401629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527636+0000","last_change":"2024-03-24T02:58:36.978649+0000","last_active":"2024-03-24T02:58:43.527636+0000","last_peered":"2024-03-24T02:58:43.527636+0000","last_clean":"2024-03-24T02:58:43.527636+0000","last_became_active":"2024-03-24T02:58:36.978136+0000","last_became_peered":"2024-03-24T02:58:36.978136+0000","last_unstale":"2024-03-24T02:58:43.527636+0000","last_undegraded":"2024-03-24T02:58:43.527636+0000","last_fullsized":"2024-03-24T02:58:43.527636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:43:55.598250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:56.056706+0000","last_change":"2024-03-24T02:58:36.138886+0000","last_active":"2024-03-24T02:58:56.056706+0000","last_peered":"2024-03-24T02:58:56.056706+0000","last_clean":"2024-03-24T02:58:56.056706+0000","last_became_active":"2024-03-24T02:58:36.138582+0000","last_became_peered":"2024-03-24T02:58:36.138582+0000","last_unstale":"2024-03-24T02:58:56.056706+0000","last_undegraded":"2024-03-24T02:58:56.056706+0000","last_fullsized":"2024-03-24T02:58:56.056706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:17:48.129273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527701+0000","last_change":"2024-03-24T02:58:41.000477+0000","last_active":"2024-03-24T02:58:43.527701+0000","last_peered":"2024-03-24T02:58:43.527701+0000","last_clean":"2024-03-24T02:58:43.527701+0000","last_became_active":"2024-03-24T02:58:41.000157+0000","last_became_peered":"2024-03-24T02:58:41.000157+0000","last_unstale":"2024-03-24T02:58:43.527701+0000","last_undegraded":"2024-03-24T02:58:43.527701+0000","last_fullsized":"2024-03-24T02:58:43.527701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:55:44.870347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986599+0000","last_change":"2024-03-24T02:58:36.135524+0000","last_active":"2024-03-24T02:58:41.986599+0000","last_peered":"2024-03-24T02:58:41.986599+0000","last_clean":"2024-03-24T02:58:41.986599+0000","last_became_active":"2024-03-24T02:58:36.135170+0000","last_became_peered":"2024-03-24T02:58:36.135170+0000","last_unstale":"2024-03-24T02:58:41.986599+0000","last_undegraded":"2024-03-24T02:58:41.986599+0000","last_fullsized":"2024-03-24T02:58:41.986599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:56:52.737193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528249+0000","last_change":"2024-03-24T02:58:36.969887+0000","last_active":"2024-03-24T02:58:43.528249+0000","last_peered":"2024-03-24T02:58:43.528249+0000","last_clean":"2024-03-24T02:58:43.528249+0000","last_became_active":"2024-03-24T02:58:36.969510+0000","last_became_peered":"2024-03-24T02:58:36.969510+0000","last_unstale":"2024-03-24T02:58:43.528249+0000","last_undegraded":"2024-03-24T02:58:43.528249+0000","last_fullsized":"2024-03-24T02:58:43.528249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:55:58.135452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.026760+0000","last_change":"2024-03-24T02:58:40.996393+0000","last_active":"2024-03-24T02:58:43.026760+0000","last_peered":"2024-03-24T02:58:43.026760+0000","last_clean":"2024-03-24T02:58:43.026760+0000","last_became_active":"2024-03-24T02:58:40.995967+0000","last_became_peered":"2024-03-24T02:58:40.995967+0000","last_unstale":"2024-03-24T02:58:43.026760+0000","last_undegraded":"2024-03-24T02:58:43.026760+0000","last_fullsized":"2024-03-24T02:58:43.026760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:14:51.160730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:56.058239+0000","last_change":"2024-03-24T02:58:36.135033+0000","last_active":"2024-03-24T02:58:56.058239+0000","last_peered":"2024-03-24T02:58:56.058239+0000","last_clean":"2024-03-24T02:58:56.058239+0000","last_became_active":"2024-03-24T02:58:36.134713+0000","last_became_peered":"2024-03-24T02:58:36.134713+0000","last_unstale":"2024-03-24T02:58:56.058239+0000","last_undegraded":"2024-03-24T02:58:56.058239+0000","last_fullsized":"2024-03-24T02:58:56.058239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:05:55.777016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.027332+0000","last_change":"2024-03-24T02:58:36.984024+0000","last_active":"2024-03-24T02:58:43.027332+0000","last_peered":"2024-03-24T02:58:43.027332+0000","last_clean":"2024-03-24T02:58:43.027332+0000","last_became_active":"2024-03-24T02:58:36.983659+0000","last_became_peered":"2024-03-24T02:58:36.983659+0000","last_unstale":"2024-03-24T02:58:43.027332+0000","last_undegraded":"2024-03-24T02:58:43.027332+0000","last_fullsized":"2024-03-24T02:58:43.027332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:47:59.900749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575406+0000","last_change":"2024-03-24T02:58:40.991181+0000","last_active":"2024-03-24T02:58:42.575406+0000","last_peered":"2024-03-24T02:58:42.575406+0000","last_clean":"2024-03-24T02:58:42.575406+0000","last_became_active":"2024-03-24T02:58:40.990755+0000","last_became_peered":"2024-03-24T02:58:40.990755+0000","last_unstale":"2024-03-24T02:58:42.575406+0000","last_undegraded":"2024-03-24T02:58:42.575406+0000","last_fullsized":"2024-03-24T02:58:42.575406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:57:05.480364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:58.083378+0000","last_change":"2024-03-24T02:58:36.123100+0000","last_active":"2024-03-24T02:58:58.083378+0000","last_peered":"2024-03-24T02:58:58.083378+0000","last_clean":"2024-03-24T02:58:58.083378+0000","last_became_active":"2024-03-24T02:58:36.122613+0000","last_became_peered":"2024-03-24T02:58:36.122613+0000","last_unstale":"2024-03-24T02:58:58.083378+0000","last_undegraded":"2024-03-24T02:58:58.083378+0000","last_fullsized":"2024-03-24T02:58:58.083378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:31:23.954532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528065+0000","last_change":"2024-03-24T02:58:36.977010+0000","last_active":"2024-03-24T02:58:43.528065+0000","last_peered":"2024-03-24T02:58:43.528065+0000","last_clean":"2024-03-24T02:58:43.528065+0000","last_became_active":"2024-03-24T02:58:36.976499+0000","last_became_peered":"2024-03-24T02:58:36.976499+0000","last_unstale":"2024-03-24T02:58:43.528065+0000","last_undegraded":"2024-03-24T02:58:43.528065+0000","last_fullsized":"2024-03-24T02:58:43.528065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:18:33.754474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528841+0000","last_change":"2024-03-24T02:58:40.991043+0000","last_active":"2024-03-24T02:58:43.528841+0000","last_peered":"2024-03-24T02:58:43.528841+0000","last_clean":"2024-03-24T02:58:43.528841+0000","last_became_active":"2024-03-24T02:58:40.990549+0000","last_became_peered":"2024-03-24T02:58:40.990549+0000","last_unstale":"2024-03-24T02:58:43.528841+0000","last_undegraded":"2024-03-24T02:58:43.528841+0000","last_fullsized":"2024-03-24T02:58:43.528841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:20:34.014398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.051343+0000","last_change":"2024-03-24T02:58:36.127420+0000","last_active":"2024-03-24T02:58:55.051343+0000","last_peered":"2024-03-24T02:58:55.051343+0000","last_clean":"2024-03-24T02:58:55.051343+0000","last_became_active":"2024-03-24T02:58:36.126995+0000","last_became_peered":"2024-03-24T02:58:36.126995+0000","last_unstale":"2024-03-24T02:58:55.051343+0000","last_undegraded":"2024-03-24T02:58:55.051343+0000","last_fullsized":"2024-03-24T02:58:55.051343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:01:52.250569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986120+0000","last_change":"2024-03-24T02:58:36.971978+0000","last_active":"2024-03-24T02:58:41.986120+0000","last_peered":"2024-03-24T02:58:41.986120+0000","last_clean":"2024-03-24T02:58:41.986120+0000","last_became_active":"2024-03-24T02:58:36.969812+0000","last_became_peered":"2024-03-24T02:58:36.969812+0000","last_unstale":"2024-03-24T02:58:41.986120+0000","last_undegraded":"2024-03-24T02:58:41.986120+0000","last_fullsized":"2024-03-24T02:58:41.986120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:44:38.573916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.986527+0000","last_change":"2024-03-24T02:58:40.994515+0000","last_active":"2024-03-24T02:58:41.986527+0000","last_peered":"2024-03-24T02:58:41.986527+0000","last_clean":"2024-03-24T02:58:41.986527+0000","last_became_active":"2024-03-24T02:58:40.994113+0000","last_became_peered":"2024-03-24T02:58:40.994113+0000","last_unstale":"2024-03-24T02:58:41.986527+0000","last_undegraded":"2024-03-24T02:58:41.986527+0000","last_fullsized":"2024-03-24T02:58:41.986527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:21:58.972660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:57.066384+0000","last_change":"2024-03-24T02:58:36.136590+0000","last_active":"2024-03-24T02:58:57.066384+0000","last_peered":"2024-03-24T02:58:57.066384+0000","last_clean":"2024-03-24T02:58:57.066384+0000","last_became_active":"2024-03-24T02:58:36.136248+0000","last_became_peered":"2024-03-24T02:58:36.136248+0000","last_unstale":"2024-03-24T02:58:57.066384+0000","last_undegraded":"2024-03-24T02:58:57.066384+0000","last_fullsized":"2024-03-24T02:58:57.066384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:20:50.738185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527379+0000","last_change":"2024-03-24T02:58:36.977145+0000","last_active":"2024-03-24T02:58:43.527379+0000","last_peered":"2024-03-24T02:58:43.527379+0000","last_clean":"2024-03-24T02:58:43.527379+0000","last_became_active":"2024-03-24T02:58:36.975835+0000","last_became_peered":"2024-03-24T02:58:36.975835+0000","last_unstale":"2024-03-24T02:58:43.527379+0000","last_undegraded":"2024-03-24T02:58:43.527379+0000","last_fullsized":"2024-03-24T02:58:43.527379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:33:30.150675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:44.293204+0000","last_change":"2024-03-24T02:58:41.005505+0000","last_active":"2024-03-24T02:58:44.293204+0000","last_peered":"2024-03-24T02:58:44.293204+0000","last_clean":"2024-03-24T02:58:44.293204+0000","last_became_active":"2024-03-24T02:58:41.005063+0000","last_became_peered":"2024-03-24T02:58:41.005063+0000","last_unstale":"2024-03-24T02:58:44.293204+0000","last_undegraded":"2024-03-24T02:58:44.293204+0000","last_fullsized":"2024-03-24T02:58:44.293204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:22:31.947209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:57.066136+0000","last_change":"2024-03-24T02:58:36.130936+0000","last_active":"2024-03-24T02:58:57.066136+0000","last_peered":"2024-03-24T02:58:57.066136+0000","last_clean":"2024-03-24T02:58:57.066136+0000","last_became_active":"2024-03-24T02:58:36.130581+0000","last_became_peered":"2024-03-24T02:58:36.130581+0000","last_unstale":"2024-03-24T02:58:57.066136+0000","last_undegraded":"2024-03-24T02:58:57.066136+0000","last_fullsized":"2024-03-24T02:58:57.066136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:52.958392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.027156+0000","last_change":"2024-03-24T02:58:36.977062+0000","last_active":"2024-03-24T02:58:43.027156+0000","last_peered":"2024-03-24T02:58:43.027156+0000","last_clean":"2024-03-24T02:58:43.027156+0000","last_became_active":"2024-03-24T02:58:36.976675+0000","last_became_peered":"2024-03-24T02:58:36.976675+0000","last_unstale":"2024-03-24T02:58:43.027156+0000","last_undegraded":"2024-03-24T02:58:43.027156+0000","last_fullsized":"2024-03-24T02:58:43.027156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:37:31.955379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528832+0000","last_change":"2024-03-24T02:58:40.996227+0000","last_active":"2024-03-24T02:58:43.528832+0000","last_peered":"2024-03-24T02:58:43.528832+0000","last_clean":"2024-03-24T02:58:43.528832+0000","last_became_active":"2024-03-24T02:58:40.995394+0000","last_became_peered":"2024-03-24T02:58:40.995394+0000","last_unstale":"2024-03-24T02:58:43.528832+0000","last_undegraded":"2024-03-24T02:58:43.528832+0000","last_fullsized":"2024-03-24T02:58:43.528832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:00:04.370712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:55.050513+0000","last_change":"2024-03-24T02:58:36.135777+0000","last_active":"2024-03-24T02:58:55.050513+0000","last_peered":"2024-03-24T02:58:55.050513+0000","last_clean":"2024-03-24T02:58:55.050513+0000","last_became_active":"2024-03-24T02:58:36.135398+0000","last_became_peered":"2024-03-24T02:58:36.135398+0000","last_unstale":"2024-03-24T02:58:55.050513+0000","last_undegraded":"2024-03-24T02:58:55.050513+0000","last_fullsized":"2024-03-24T02:58:55.050513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:11:01.641323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.989415+0000","last_change":"2024-03-24T02:58:36.981004+0000","last_active":"2024-03-24T02:58:41.989415+0000","last_peered":"2024-03-24T02:58:41.989415+0000","last_clean":"2024-03-24T02:58:41.989415+0000","last_became_active":"2024-03-24T02:58:36.980258+0000","last_became_peered":"2024-03-24T02:58:36.980258+0000","last_unstale":"2024-03-24T02:58:41.989415+0000","last_undegraded":"2024-03-24T02:58:41.989415+0000","last_fullsized":"2024-03-24T02:58:41.989415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:21:22.479122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528206+0000","last_change":"2024-03-24T02:58:40.994636+0000","last_active":"2024-03-24T02:58:43.528206+0000","last_peered":"2024-03-24T02:58:43.528206+0000","last_clean":"2024-03-24T02:58:43.528206+0000","last_became_active":"2024-03-24T02:58:40.994122+0000","last_became_peered":"2024-03-24T02:58:40.994122+0000","last_unstale":"2024-03-24T02:58:43.528206+0000","last_undegraded":"2024-03-24T02:58:43.528206+0000","last_fullsized":"2024-03-24T02:58:43.528206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:55:56.707430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:59.095540+0000","last_change":"2024-03-24T02:58:36.134528+0000","last_active":"2024-03-24T02:58:59.095540+0000","last_peered":"2024-03-24T02:58:59.095540+0000","last_clean":"2024-03-24T02:58:59.095540+0000","last_became_active":"2024-03-24T02:58:36.134158+0000","last_became_peered":"2024-03-24T02:58:36.134158+0000","last_unstale":"2024-03-24T02:58:59.095540+0000","last_undegraded":"2024-03-24T02:58:59.095540+0000","last_fullsized":"2024-03-24T02:58:59.095540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:50:31.582575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.989380+0000","last_change":"2024-03-24T02:58:36.984545+0000","last_active":"2024-03-24T02:58:41.989380+0000","last_peered":"2024-03-24T02:58:41.989380+0000","last_clean":"2024-03-24T02:58:41.989380+0000","last_became_active":"2024-03-24T02:58:36.982843+0000","last_became_peered":"2024-03-24T02:58:36.982843+0000","last_unstale":"2024-03-24T02:58:41.989380+0000","last_undegraded":"2024-03-24T02:58:41.989380+0000","last_fullsized":"2024-03-24T02:58:41.989380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:50:04.481097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527531+0000","last_change":"2024-03-24T02:58:41.000190+0000","last_active":"2024-03-24T02:58:43.527531+0000","last_peered":"2024-03-24T02:58:43.527531+0000","last_clean":"2024-03-24T02:58:43.527531+0000","last_became_active":"2024-03-24T02:58:40.999767+0000","last_became_peered":"2024-03-24T02:58:40.999767+0000","last_unstale":"2024-03-24T02:58:43.527531+0000","last_undegraded":"2024-03-24T02:58:43.527531+0000","last_fullsized":"2024-03-24T02:58:43.527531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:31:44.927277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:57.066482+0000","last_change":"2024-03-24T02:58:36.133118+0000","last_active":"2024-03-24T02:58:57.066482+0000","last_peered":"2024-03-24T02:58:57.066482+0000","last_clean":"2024-03-24T02:58:57.066482+0000","last_became_active":"2024-03-24T02:58:36.132747+0000","last_became_peered":"2024-03-24T02:58:36.132747+0000","last_unstale":"2024-03-24T02:58:57.066482+0000","last_undegraded":"2024-03-24T02:58:57.066482+0000","last_fullsized":"2024-03-24T02:58:57.066482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:11:15.665601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.988973+0000","last_change":"2024-03-24T02:58:36.985489+0000","last_active":"2024-03-24T02:58:41.988973+0000","last_peered":"2024-03-24T02:58:41.988973+0000","last_clean":"2024-03-24T02:58:41.988973+0000","last_became_active":"2024-03-24T02:58:36.984421+0000","last_became_peered":"2024-03-24T02:58:36.984421+0000","last_unstale":"2024-03-24T02:58:41.988973+0000","last_undegraded":"2024-03-24T02:58:41.988973+0000","last_fullsized":"2024-03-24T02:58:41.988973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:57:03.004888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527670+0000","last_change":"2024-03-24T02:58:40.991456+0000","last_active":"2024-03-24T02:58:43.527670+0000","last_peered":"2024-03-24T02:58:43.527670+0000","last_clean":"2024-03-24T02:58:43.527670+0000","last_became_active":"2024-03-24T02:58:40.990973+0000","last_became_peered":"2024-03-24T02:58:40.990973+0000","last_unstale":"2024-03-24T02:58:43.527670+0000","last_undegraded":"2024-03-24T02:58:43.527670+0000","last_fullsized":"2024-03-24T02:58:43.527670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:36:01.276988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:58.081728+0000","last_change":"2024-03-24T02:58:36.130967+0000","last_active":"2024-03-24T02:58:58.081728+0000","last_peered":"2024-03-24T02:58:58.081728+0000","last_clean":"2024-03-24T02:58:58.081728+0000","last_became_active":"2024-03-24T02:58:36.130657+0000","last_became_peered":"2024-03-24T02:58:36.130657+0000","last_unstale":"2024-03-24T02:58:58.081728+0000","last_undegraded":"2024-03-24T02:58:58.081728+0000","last_fullsized":"2024-03-24T02:58:58.081728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:26:20.426922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:41.989135+0000","last_change":"2024-03-24T02:58:36.982297+0000","last_active":"2024-03-24T02:58:41.989135+0000","last_peered":"2024-03-24T02:58:41.989135+0000","last_clean":"2024-03-24T02:58:41.989135+0000","last_became_active":"2024-03-24T02:58:36.981792+0000","last_became_peered":"2024-03-24T02:58:36.981792+0000","last_unstale":"2024-03-24T02:58:41.989135+0000","last_undegraded":"2024-03-24T02:58:41.989135+0000","last_fullsized":"2024-03-24T02:58:41.989135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:03:07.528142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527432+0000","last_change":"2024-03-24T02:58:36.979815+0000","last_active":"2024-03-24T02:58:43.527432+0000","last_peered":"2024-03-24T02:58:43.527432+0000","last_clean":"2024-03-24T02:58:43.527432+0000","last_became_active":"2024-03-24T02:58:36.979467+0000","last_became_peered":"2024-03-24T02:58:36.979467+0000","last_unstale":"2024-03-24T02:58:43.527432+0000","last_undegraded":"2024-03-24T02:58:43.527432+0000","last_fullsized":"2024-03-24T02:58:43.527432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:28.056630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575292+0000","last_change":"2024-03-24T02:58:36.974644+0000","last_active":"2024-03-24T02:58:42.575292+0000","last_peered":"2024-03-24T02:58:42.575292+0000","last_clean":"2024-03-24T02:58:42.575292+0000","last_became_active":"2024-03-24T02:58:36.974276+0000","last_became_peered":"2024-03-24T02:58:36.974276+0000","last_unstale":"2024-03-24T02:58:42.575292+0000","last_undegraded":"2024-03-24T02:58:42.575292+0000","last_fullsized":"2024-03-24T02:58:42.575292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:04.606926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575217+0000","last_change":"2024-03-24T02:58:36.978207+0000","last_active":"2024-03-24T02:58:42.575217+0000","last_peered":"2024-03-24T02:58:42.575217+0000","last_clean":"2024-03-24T02:58:42.575217+0000","last_became_active":"2024-03-24T02:58:36.977760+0000","last_became_peered":"2024-03-24T02:58:36.977760+0000","last_unstale":"2024-03-24T02:58:42.575217+0000","last_undegraded":"2024-03-24T02:58:42.575217+0000","last_fullsized":"2024-03-24T02:58:42.575217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:43:51.603963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527813+0000","last_change":"2024-03-24T02:58:36.975740+0000","last_active":"2024-03-24T02:58:43.527813+0000","last_peered":"2024-03-24T02:58:43.527813+0000","last_clean":"2024-03-24T02:58:43.527813+0000","last_became_active":"2024-03-24T02:58:36.975316+0000","last_became_peered":"2024-03-24T02:58:36.975316+0000","last_unstale":"2024-03-24T02:58:43.527813+0000","last_undegraded":"2024-03-24T02:58:43.527813+0000","last_fullsized":"2024-03-24T02:58:43.527813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:21:33.579169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528364+0000","last_change":"2024-03-24T02:58:36.977261+0000","last_active":"2024-03-24T02:58:43.528364+0000","last_peered":"2024-03-24T02:58:43.528364+0000","last_clean":"2024-03-24T02:58:43.528364+0000","last_became_active":"2024-03-24T02:58:36.974644+0000","last_became_peered":"2024-03-24T02:58:36.974644+0000","last_unstale":"2024-03-24T02:58:43.528364+0000","last_undegraded":"2024-03-24T02:58:43.528364+0000","last_fullsized":"2024-03-24T02:58:43.528364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:34:50.968770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527857+0000","last_change":"2024-03-24T02:58:36.981223+0000","last_active":"2024-03-24T02:58:43.527857+0000","last_peered":"2024-03-24T02:58:43.527857+0000","last_clean":"2024-03-24T02:58:43.527857+0000","last_became_active":"2024-03-24T02:58:36.979728+0000","last_became_peered":"2024-03-24T02:58:36.979728+0000","last_unstale":"2024-03-24T02:58:43.527857+0000","last_undegraded":"2024-03-24T02:58:43.527857+0000","last_fullsized":"2024-03-24T02:58:43.527857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:08:41.300442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:42.575700+0000","last_change":"2024-03-24T02:58:36.972385+0000","last_active":"2024-03-24T02:58:42.575700+0000","last_peered":"2024-03-24T02:58:42.575700+0000","last_clean":"2024-03-24T02:58:42.575700+0000","last_became_active":"2024-03-24T02:58:36.972018+0000","last_became_peered":"2024-03-24T02:58:36.972018+0000","last_unstale":"2024-03-24T02:58:42.575700+0000","last_undegraded":"2024-03-24T02:58:42.575700+0000","last_fullsized":"2024-03-24T02:58:42.575700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:02:22.529198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.027444+0000","last_change":"2024-03-24T02:58:36.975284+0000","last_active":"2024-03-24T02:58:43.027444+0000","last_peered":"2024-03-24T02:58:43.027444+0000","last_clean":"2024-03-24T02:58:43.027444+0000","last_became_active":"2024-03-24T02:58:36.974942+0000","last_became_peered":"2024-03-24T02:58:36.974942+0000","last_unstale":"2024-03-24T02:58:43.027444+0000","last_undegraded":"2024-03-24T02:58:43.027444+0000","last_fullsized":"2024-03-24T02:58:43.027444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:58:50.093347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527780+0000","last_change":"2024-03-24T02:58:36.977256+0000","last_active":"2024-03-24T02:58:43.527780+0000","last_peered":"2024-03-24T02:58:43.527780+0000","last_clean":"2024-03-24T02:58:43.527780+0000","last_became_active":"2024-03-24T02:58:36.976888+0000","last_became_peered":"2024-03-24T02:58:36.976888+0000","last_unstale":"2024-03-24T02:58:43.527780+0000","last_undegraded":"2024-03-24T02:58:43.527780+0000","last_fullsized":"2024-03-24T02:58:43.527780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:18:19.449590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.026867+0000","last_change":"2024-03-24T02:58:36.981413+0000","last_active":"2024-03-24T02:58:43.026867+0000","last_peered":"2024-03-24T02:58:43.026867+0000","last_clean":"2024-03-24T02:58:43.026867+0000","last_became_active":"2024-03-24T02:58:36.980346+0000","last_became_peered":"2024-03-24T02:58:36.980346+0000","last_unstale":"2024-03-24T02:58:43.026867+0000","last_undegraded":"2024-03-24T02:58:43.026867+0000","last_fullsized":"2024-03-24T02:58:43.026867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:17:57.332228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.528552+0000","last_change":"2024-03-24T02:58:36.984264+0000","last_active":"2024-03-24T02:58:43.528552+0000","last_peered":"2024-03-24T02:58:43.528552+0000","last_clean":"2024-03-24T02:58:43.528552+0000","last_became_active":"2024-03-24T02:58:36.983905+0000","last_became_peered":"2024-03-24T02:58:36.983905+0000","last_unstale":"2024-03-24T02:58:43.528552+0000","last_undegraded":"2024-03-24T02:58:43.528552+0000","last_fullsized":"2024-03-24T02:58:43.528552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:25:17.872320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527780+0000","last_change":"2024-03-24T02:58:36.975939+0000","last_active":"2024-03-24T02:58:43.527780+0000","last_peered":"2024-03-24T02:58:43.527780+0000","last_clean":"2024-03-24T02:58:43.527780+0000","last_became_active":"2024-03-24T02:58:36.975477+0000","last_became_peered":"2024-03-24T02:58:36.975477+0000","last_unstale":"2024-03-24T02:58:43.527780+0000","last_undegraded":"2024-03-24T02:58:43.527780+0000","last_fullsized":"2024-03-24T02:58:43.527780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:46:43.603391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-24T02:58:43.527568+0000","last_change":"2024-03-24T02:58:36.980674+0000","last_active":"2024-03-24T02:58:43.527568+0000","last_peered":"2024-03-24T02:58:43.527568+0000","last_clean":"2024-03-24T02:58:43.527568+0000","last_became_active":"2024-03-24T02:58:36.980278+0000","last_became_peered":"2024-03-24T02:58:36.980278+0000","last_unstale":"2024-03-24T02:58:43.527568+0000","last_undegraded":"2024-03-24T02:58:43.527568+0000","last_fullsized":"2024-03-24T02:58:43.527568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:50:01.719362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10172,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":167,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":352256,"data_stored":56574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":478407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1073127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":478033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":477819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1613824,"data_stored":1075212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":477937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":481057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":478693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":511324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":477499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":478306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36920,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708232,"statfs":{"total":95995035648,"available":95957229568,"internally_reserved":0,"allocated":1622016,"data_stored":1070403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":34109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":5544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-24T02:59:00.895 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-24T02:59:00.895 INFO:tasks.cephadm:Setting up client nodes... 2024-03-24T02:59:00.896 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-24T02:59:01.184 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:01.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:01 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/890970100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:59:01.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:01 smithi017 ceph-mon[31926]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T02:59:01.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:01 smithi017 ceph-mon[31926]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T02:59:01.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:01 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/890970100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:59:01.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:01 smithi188 ceph-mon[36206]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T02:59:01.550 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:01 smithi188 ceph-mon[36206]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T02:59:01.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:01 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/890970100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:59:01.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:01 smithi019 ceph-mon[35944]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T02:59:01.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:01 smithi019 ceph-mon[35944]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T02:59:02.451 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:02 smithi017 ceph-mon[31926]: pgmap v305: 129 pgs: 129 active+clean; 588 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.9 KiB/s wr, 16 op/s 2024-03-24T02:59:02.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:02 smithi188 ceph-mon[36206]: pgmap v305: 129 pgs: 129 active+clean; 588 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.9 KiB/s wr, 16 op/s 2024-03-24T02:59:02.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:02 smithi019 ceph-mon[35944]: pgmap v305: 129 pgs: 129 active+clean; 588 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.9 KiB/s wr, 16 op/s 2024-03-24T02:59:03.047 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 02:59:02 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i[67156]: 2024-03-24T02:59:02.749+0000 7f913f57f700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-24T02:59:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.134+0000 7f49e7b3c700 1 -- 172.21.15.17:0/3462711125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49e0103400 msgr2=0x7f49e0103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.134+0000 7f49e7b3c700 1 --2- 172.21.15.17:0/3462711125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49e0103400 0x7f49e0103860 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f49d00099c0 tx=0x7f49d0009cd0 comp rx=0 tx=0).stop 2024-03-24T02:59:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.134+0000 7f49e7b3c700 1 -- 172.21.15.17:0/3462711125 shutdown_connections 2024-03-24T02:59:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.134+0000 7f49e7b3c700 1 --2- 172.21.15.17:0/3462711125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e0103da0 0x7f49e010a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.134+0000 7f49e7b3c700 1 --2- 172.21.15.17:0/3462711125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e0102200 0x7f49e0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.134+0000 7f49e7b3c700 1 --2- 172.21.15.17:0/3462711125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49e0103400 0x7f49e0103860 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.134+0000 7f49e7b3c700 1 -- 172.21.15.17:0/3462711125 >> 172.21.15.17:0/3462711125 conn(0x7f49e00fd7a0 msgr2=0x7f49e00ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.134+0000 7f49e7b3c700 1 -- 172.21.15.17:0/3462711125 shutdown_connections 2024-03-24T02:59:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.134+0000 7f49e7b3c700 1 -- 172.21.15.17:0/3462711125 wait complete. 2024-03-24T02:59:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e7b3c700 1 Processor -- start 2024-03-24T02:59:03.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e7b3c700 1 -- start start 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e7b3c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e0102200 0x7f49e01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e7b3c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49e0103400 0x7f49e0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e7b3c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e0103da0 0x7f49e019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e7b3c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e010ddc0 con 0x7f49e0103400 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e7b3c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e010dc40 con 0x7f49e0103da0 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e7b3c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e010df40 con 0x7f49e0102200 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e50d7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49e0103400 0x7f49e0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e50d7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49e0103400 0x7f49e0198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59272/0 (socket says 172.21.15.17:59272) 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e50d7700 1 -- 172.21.15.17:0/2639490193 learned_addr learned my addr 172.21.15.17:0/2639490193 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e58d8700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e0102200 0x7f49e01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.135+0000 7f49e60d9700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e0103da0 0x7f49e019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.136+0000 7f49e50d7700 1 -- 172.21.15.17:0/2639490193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e0102200 msgr2=0x7f49e01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.136+0000 7f49e50d7700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e0102200 0x7f49e01982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:03.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.136+0000 7f49e50d7700 1 -- 172.21.15.17:0/2639490193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e0103da0 msgr2=0x7f49e019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.136+0000 7f49e50d7700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e0103da0 0x7f49e019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.136+0000 7f49e50d7700 1 -- 172.21.15.17:0/2639490193 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e019ff40 con 0x7f49e0103400 2024-03-24T02:59:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.136+0000 7f49e58d8700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e0102200 0x7f49e01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.136+0000 7f49e50d7700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49e0103400 0x7f49e0198820 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f49d0005910 tx=0x7f49d000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.136+0000 7f49d6ffd700 1 -- 172.21.15.17:0/2639490193 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0009e60 con 0x7f49e0103400 2024-03-24T02:59:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.136+0000 7f49d6ffd700 1 -- 172.21.15.17:0/2639490193 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49d0003960 con 0x7f49e0103400 2024-03-24T02:59:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.136+0000 7f49e7b3c700 1 -- 172.21.15.17:0/2639490193 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e01a01d0 con 0x7f49e0103400 2024-03-24T02:59:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.136+0000 7f49d6ffd700 1 -- 172.21.15.17:0/2639490193 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d00173f0 con 0x7f49e0103400 2024-03-24T02:59:03.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.136+0000 7f49e7b3c700 1 -- 172.21.15.17:0/2639490193 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e01a0620 con 0x7f49e0103400 2024-03-24T02:59:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.137+0000 7f49e7b3c700 1 -- 172.21.15.17:0/2639490193 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e00506a0 con 0x7f49e0103400 2024-03-24T02:59:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.138+0000 7f49d6ffd700 1 -- 172.21.15.17:0/2639490193 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f49d0003b10 con 0x7f49e0103400 2024-03-24T02:59:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.139+0000 7f49d6ffd700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f49cc0a2a50 0x7f49cc0a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.139+0000 7f49e58d8700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f49cc0a2a50 0x7f49cc0a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.139+0000 7f49d6ffd700 1 -- 172.21.15.17:0/2639490193 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f49d00cfc50 con 0x7f49e0103400 2024-03-24T02:59:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.139+0000 7f49e58d8700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f49cc0a2a50 0x7f49cc0a4f10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f49e0103260 tx=0x7f49dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:03.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.143+0000 7f49d6ffd700 1 -- 172.21.15.17:0/2639490193 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f49d0013070 con 0x7f49e0103400 2024-03-24T02:59:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.432+0000 7f49e7b3c700 1 -- 172.21.15.17:0/2639490193 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f49e002db50 con 0x7f49e0103400 2024-03-24T02:59:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.434+0000 7f49d6ffd700 1 -- 172.21.15.17:0/2639490193 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f49e002db50 con 0x7f49e0103400 2024-03-24T02:59:03.433 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-24T02:59:03.433 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD3lv9lGjHUGRAAGwWK4Rr4S3JZ+9tpQq/YlA== 2024-03-24T02:59:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.437+0000 7f49e7b3c700 1 -- 172.21.15.17:0/2639490193 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f49cc0a2a50 msgr2=0x7f49cc0a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.437+0000 7f49e7b3c700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f49cc0a2a50 0x7f49cc0a4f10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f49e0103260 tx=0x7f49dc009260 comp rx=0 tx=0).stop 2024-03-24T02:59:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.437+0000 7f49e7b3c700 1 -- 172.21.15.17:0/2639490193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49e0103400 msgr2=0x7f49e0198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.437+0000 7f49e7b3c700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49e0103400 0x7f49e0198820 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f49d0005910 tx=0x7f49d000f690 comp rx=0 tx=0).stop 2024-03-24T02:59:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.437+0000 7f49e7b3c700 1 -- 172.21.15.17:0/2639490193 shutdown_connections 2024-03-24T02:59:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.437+0000 7f49e7b3c700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f49cc0a2a50 0x7f49cc0a4f10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.437+0000 7f49e7b3c700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e0103da0 0x7f49e019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.437+0000 7f49e7b3c700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e0102200 0x7f49e01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.437+0000 7f49e7b3c700 1 --2- 172.21.15.17:0/2639490193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49e0103400 0x7f49e0198820 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.437+0000 7f49e7b3c700 1 -- 172.21.15.17:0/2639490193 >> 172.21.15.17:0/2639490193 conn(0x7f49e00fd7a0 msgr2=0x7f49e0106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.437+0000 7f49e7b3c700 1 -- 172.21.15.17:0/2639490193 shutdown_connections 2024-03-24T02:59:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:03.437+0000 7f49e7b3c700 1 -- 172.21.15.17:0/2639490193 wait complete. 2024-03-24T02:59:03.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:03 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2639490193' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T02:59:03.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:03 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2639490193' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T02:59:03.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:03 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2639490193' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T02:59:03.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:03 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2639490193' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T02:59:03.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:03 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2639490193' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T02:59:03.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:03 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2639490193' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T02:59:04.191 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T02:59:04.191 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-24T02:59:04.191 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-24T02:59:04.235 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-24T02:59:04.236 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-24T02:59:04.236 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-24T02:59:04.521 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:04.755 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:04 smithi017 ceph-mon[31926]: pgmap v306: 129 pgs: 129 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.8 KiB/s wr, 21 op/s 2024-03-24T02:59:04.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:04 smithi188 ceph-mon[36206]: pgmap v306: 129 pgs: 129 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.8 KiB/s wr, 21 op/s 2024-03-24T02:59:04.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:04 smithi019 ceph-mon[35944]: pgmap v306: 129 pgs: 129 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.8 KiB/s wr, 21 op/s 2024-03-24T02:59:06.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:06 smithi188 ceph-mon[36206]: pgmap v307: 129 pgs: 129 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.5 KiB/s wr, 22 op/s 2024-03-24T02:59:06.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:06 smithi019 ceph-mon[35944]: pgmap v307: 129 pgs: 129 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.5 KiB/s wr, 22 op/s 2024-03-24T02:59:06.349 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:06 smithi017 ceph-mon[31926]: pgmap v307: 129 pgs: 129 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.5 KiB/s wr, 22 op/s 2024-03-24T02:59:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.349+0000 7fa1f63e5700 1 -- 172.21.15.17:0/2843314453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0102ba0 msgr2=0x7fa1f010efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.349+0000 7fa1f63e5700 1 --2- 172.21.15.17:0/2843314453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0102ba0 0x7fa1f010efe0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800b0d0 tx=0x7fa1e800b3e0 comp rx=0 tx=0).stop 2024-03-24T02:59:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.349+0000 7fa1f63e5700 1 -- 172.21.15.17:0/2843314453 shutdown_connections 2024-03-24T02:59:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.349+0000 7fa1f63e5700 1 --2- 172.21.15.17:0/2843314453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0102ba0 0x7fa1f010efe0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.349+0000 7fa1f63e5700 1 --2- 172.21.15.17:0/2843314453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01081c0 0x7fa1f01085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.349+0000 7fa1f63e5700 1 --2- 172.21.15.17:0/2843314453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1f01021e0 0x7fa1f0102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.349+0000 7fa1f63e5700 1 -- 172.21.15.17:0/2843314453 >> 172.21.15.17:0/2843314453 conn(0x7fa1f00792c0 msgr2=0x7fa1f00796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.349+0000 7fa1f63e5700 1 -- 172.21.15.17:0/2843314453 shutdown_connections 2024-03-24T02:59:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.349+0000 7fa1f63e5700 1 -- 172.21.15.17:0/2843314453 wait complete. 2024-03-24T02:59:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.350+0000 7fa1f63e5700 1 Processor -- start 2024-03-24T02:59:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.350+0000 7fa1f63e5700 1 -- start start 2024-03-24T02:59:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.350+0000 7fa1f63e5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01021e0 0x7fa1f0074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.350+0000 7fa1f63e5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1f0102ba0 0x7fa1f00749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.350+0000 7fa1f63e5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f01081c0 0x7fa1f0074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.350+0000 7fa1f63e5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f0112460 con 0x7fa1f01081c0 2024-03-24T02:59:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.350+0000 7fa1f63e5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f01122e0 con 0x7fa1f0102ba0 2024-03-24T02:59:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1f63e5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f01125e0 con 0x7fa1f01021e0 2024-03-24T02:59:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1f4982700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f01081c0 0x7fa1f0074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1effff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01021e0 0x7fa1f0074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1f4982700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f01081c0 0x7fa1f0074f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59288/0 (socket says 172.21.15.17:59288) 2024-03-24T02:59:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1effff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01021e0 0x7fa1f0074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:53176/0 (socket says 172.21.15.17:53176) 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1f4982700 1 -- 172.21.15.17:0/678641522 learned_addr learned my addr 172.21.15.17:0/678641522 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1ef7fe700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1f0102ba0 0x7fa1f00749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1effff700 1 -- 172.21.15.17:0/678641522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1f0102ba0 msgr2=0x7fa1f00749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1effff700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1f0102ba0 0x7fa1f00749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1effff700 1 -- 172.21.15.17:0/678641522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f01081c0 msgr2=0x7fa1f0074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1effff700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f01081c0 0x7fa1f0074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1effff700 1 -- 172.21.15.17:0/678641522 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f01a6d70 con 0x7fa1f01021e0 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1f4982700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f01081c0 0x7fa1f0074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1effff700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01021e0 0x7fa1f0074490 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000e9f0 tx=0x7fa1e000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1ed7fa700 1 -- 172.21.15.17:0/678641522 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0004220 con 0x7fa1f01021e0 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1ed7fa700 1 -- 172.21.15.17:0/678641522 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0004d10 con 0x7fa1f01021e0 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1f63e5700 1 -- 172.21.15.17:0/678641522 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f01a7060 con 0x7fa1f01021e0 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.351+0000 7fa1ed7fa700 1 -- 172.21.15.17:0/678641522 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0005020 con 0x7fa1f01021e0 2024-03-24T02:59:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.352+0000 7fa1f63e5700 1 -- 172.21.15.17:0/678641522 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f01a7550 con 0x7fa1f01021e0 2024-03-24T02:59:06.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.353+0000 7fa1f63e5700 1 -- 172.21.15.17:0/678641522 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f0068e50 con 0x7fa1f01021e0 2024-03-24T02:59:06.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.354+0000 7fa1ed7fa700 1 -- 172.21.15.17:0/678641522 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fa1e0004750 con 0x7fa1f01021e0 2024-03-24T02:59:06.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.354+0000 7fa1ed7fa700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa1dc0a2920 0x7fa1dc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:06.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.354+0000 7fa1ef7fe700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa1dc0a2920 0x7fa1dc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:06.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.354+0000 7fa1ed7fa700 1 -- 172.21.15.17:0/678641522 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa1e00cece0 con 0x7fa1f01021e0 2024-03-24T02:59:06.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.355+0000 7fa1ef7fe700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa1dc0a2920 0x7fa1dc0a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00798a0 tx=0x7fa1d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:06.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.361+0000 7fa1ed7fa700 1 -- 172.21.15.17:0/678641522 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa1e009b980 con 0x7fa1f01021e0 2024-03-24T02:59:06.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.629+0000 7fa1f63e5700 1 -- 172.21.15.17:0/678641522 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa1f002da30 con 0x7fa1f01021e0 2024-03-24T02:59:06.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.634+0000 7fa1ed7fa700 1 -- 172.21.15.17:0/678641522 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+262658 (secure 0 0 0) 0x7fa1e009b280 con 0x7fa1f01021e0 2024-03-24T02:59:06.637 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:59:06.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.637+0000 7fa1f63e5700 1 -- 172.21.15.17:0/678641522 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa1dc0a2920 msgr2=0x7fa1dc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:06.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.637+0000 7fa1f63e5700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa1dc0a2920 0x7fa1dc0a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00798a0 tx=0x7fa1d8009260 comp rx=0 tx=0).stop 2024-03-24T02:59:06.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.637+0000 7fa1f63e5700 1 -- 172.21.15.17:0/678641522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01021e0 msgr2=0x7fa1f0074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:06.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.637+0000 7fa1f63e5700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01021e0 0x7fa1f0074490 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000e9f0 tx=0x7fa1e000ed00 comp rx=0 tx=0).stop 2024-03-24T02:59:06.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.637+0000 7fa1f63e5700 1 -- 172.21.15.17:0/678641522 shutdown_connections 2024-03-24T02:59:06.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.637+0000 7fa1f63e5700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa1dc0a2920 0x7fa1dc0a4de0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:06.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.637+0000 7fa1f63e5700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f01081c0 0x7fa1f0074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:06.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.637+0000 7fa1f63e5700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f01021e0 0x7fa1f0074490 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:06.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.637+0000 7fa1f63e5700 1 --2- 172.21.15.17:0/678641522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1f0102ba0 0x7fa1f00749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:06.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.637+0000 7fa1f63e5700 1 -- 172.21.15.17:0/678641522 >> 172.21.15.17:0/678641522 conn(0x7fa1f00792c0 msgr2=0x7fa1f0100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:06.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.637+0000 7fa1f63e5700 1 -- 172.21.15.17:0/678641522 shutdown_connections 2024-03-24T02:59:06.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:06.637+0000 7fa1f63e5700 1 -- 172.21.15.17:0/678641522 wait complete. 2024-03-24T02:59:07.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:07 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/678641522' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T02:59:07.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:07 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/678641522' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T02:59:07.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:07 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/678641522' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T02:59:07.435 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2663792883},{"type":"v1","addr":"172.21.15.17:6801","nonce":2663792883}]},"active_addr":"172.21.15.17:6801/2663792883","active_change":"2024-03-24T02:51:29.864678+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.17:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":1698088665}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":1127409383}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":3359268740}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":934316425}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":89769044}]}]} 2024-03-24T02:59:07.440 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-24T02:59:07.440 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-24T02:59:07.440 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T02:59:07.685 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:08.056 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:08 smithi017 ceph-mon[31926]: pgmap v308: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 23 op/s 2024-03-24T02:59:08.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:08 smithi188 ceph-mon[36206]: pgmap v308: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 23 op/s 2024-03-24T02:59:08.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:08 smithi019 ceph-mon[35944]: pgmap v308: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 23 op/s 2024-03-24T02:59:09.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.470+0000 7f99f4972700 1 -- 172.21.15.17:0/3015451311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0101a30 msgr2=0x7f99f010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:09.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.470+0000 7f99f4972700 1 --2- 172.21.15.17:0/3015451311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0101a30 0x7f99f010ded0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f99e800b0a0 tx=0x7f99e800b3b0 comp rx=0 tx=0).stop 2024-03-24T02:59:09.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.470+0000 7f99f4972700 1 -- 172.21.15.17:0/3015451311 shutdown_connections 2024-03-24T02:59:09.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.470+0000 7f99f4972700 1 --2- 172.21.15.17:0/3015451311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0101a30 0x7f99f010ded0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:09.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.470+0000 7f99f4972700 1 --2- 172.21.15.17:0/3015451311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0107120 0x7f99f0107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:09.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.470+0000 7f99f4972700 1 --2- 172.21.15.17:0/3015451311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f0101070 0x7f99f01014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:09.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.470+0000 7f99f4972700 1 -- 172.21.15.17:0/3015451311 >> 172.21.15.17:0/3015451311 conn(0x7f99f00780a0 msgr2=0x7f99f00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:09.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.470+0000 7f99f4972700 1 -- 172.21.15.17:0/3015451311 shutdown_connections 2024-03-24T02:59:09.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.470+0000 7f99f4972700 1 -- 172.21.15.17:0/3015451311 wait complete. 2024-03-24T02:59:09.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99f4972700 1 Processor -- start 2024-03-24T02:59:09.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99f4972700 1 -- start start 2024-03-24T02:59:09.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99f4972700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f0101070 0x7f99f0103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:09.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99f4972700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0101a30 0x7f99f01035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:09.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99f4972700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0107120 0x7f99f0103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:09.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99f4972700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0111330 con 0x7f99f0107120 2024-03-24T02:59:09.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99f4972700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01111b0 con 0x7f99f0101070 2024-03-24T02:59:09.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99f4972700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01114b0 con 0x7f99f0101a30 2024-03-24T02:59:09.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0107120 0x7f99f0103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:09.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0107120 0x7f99f0103b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59308/0 (socket says 172.21.15.17:59308) 2024-03-24T02:59:09.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99eed9d700 1 -- 172.21.15.17:0/2362085565 learned_addr learned my addr 172.21.15.17:0/2362085565 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:09.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99edd9b700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0101a30 0x7f99f01035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:09.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99ee59c700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f0101070 0x7f99f0103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:09.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99eed9d700 1 -- 172.21.15.17:0/2362085565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0101a30 msgr2=0x7f99f01035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:09.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99eed9d700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0101a30 0x7f99f01035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:09.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99eed9d700 1 -- 172.21.15.17:0/2362085565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f0101070 msgr2=0x7f99f0103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:09.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99eed9d700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f0101070 0x7f99f0103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:09.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99eed9d700 1 -- 172.21.15.17:0/2362085565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f01a6d60 con 0x7f99f0107120 2024-03-24T02:59:09.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.471+0000 7f99ee59c700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f0101070 0x7f99f0103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:09.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.472+0000 7f99eed9d700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0107120 0x7f99f0103b10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f99e800b780 tx=0x7f99e8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:09.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.472+0000 7f99df7fe700 1 -- 172.21.15.17:0/2362085565 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8021070 con 0x7f99f0107120 2024-03-24T02:59:09.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.472+0000 7f99f4972700 1 -- 172.21.15.17:0/2362085565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f01a6ff0 con 0x7f99f0107120 2024-03-24T02:59:09.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.472+0000 7f99df7fe700 1 -- 172.21.15.17:0/2362085565 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99e8007b80 con 0x7f99f0107120 2024-03-24T02:59:09.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.472+0000 7f99df7fe700 1 -- 172.21.15.17:0/2362085565 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80052a0 con 0x7f99f0107120 2024-03-24T02:59:09.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.472+0000 7f99f4972700 1 -- 172.21.15.17:0/2362085565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f01a7550 con 0x7f99f0107120 2024-03-24T02:59:09.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.473+0000 7f99f4972700 1 -- 172.21.15.17:0/2362085565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f0068e50 con 0x7f99f0107120 2024-03-24T02:59:09.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.474+0000 7f99df7fe700 1 -- 172.21.15.17:0/2362085565 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f99e8024070 con 0x7f99f0107120 2024-03-24T02:59:09.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.474+0000 7f99df7fe700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f99d80a2920 0x7f99d80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:09.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.475+0000 7f99ee59c700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f99d80a2920 0x7f99d80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:09.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.475+0000 7f99df7fe700 1 -- 172.21.15.17:0/2362085565 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f99e800f020 con 0x7f99f0107120 2024-03-24T02:59:09.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.475+0000 7f99ee59c700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f99d80a2920 0x7f99d80a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009fd0 tx=0x7f99e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:09.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.479+0000 7f99df7fe700 1 -- 172.21.15.17:0/2362085565 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f99e802b020 con 0x7f99f0107120 2024-03-24T02:59:09.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.682+0000 7f99f4972700 1 -- 172.21.15.17:0/2362085565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99f01a77c0 con 0x7f99f0107120 2024-03-24T02:59:09.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.683+0000 7f99df7fe700 1 -- 172.21.15.17:0/2362085565 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24100 (secure 0 0 0) 0x7f99e8007d30 con 0x7f99f0107120 2024-03-24T02:59:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:59:09.684 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:58:41.346962+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:59:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.686+0000 7f99f4972700 1 -- 172.21.15.17:0/2362085565 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f99d80a2920 msgr2=0x7f99d80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.686+0000 7f99f4972700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f99d80a2920 0x7f99d80a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009fd0 tx=0x7f99e0009260 comp rx=0 tx=0).stop 2024-03-24T02:59:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.686+0000 7f99f4972700 1 -- 172.21.15.17:0/2362085565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0107120 msgr2=0x7f99f0103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.686+0000 7f99f4972700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0107120 0x7f99f0103b10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f99e800b780 tx=0x7f99e8004d00 comp rx=0 tx=0).stop 2024-03-24T02:59:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.686+0000 7f99f4972700 1 -- 172.21.15.17:0/2362085565 shutdown_connections 2024-03-24T02:59:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.686+0000 7f99f4972700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f99d80a2920 0x7f99d80a4de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.686+0000 7f99f4972700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0107120 0x7f99f0103b10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.686+0000 7f99f4972700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f0101070 0x7f99f0103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.686+0000 7f99f4972700 1 --2- 172.21.15.17:0/2362085565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0101a30 0x7f99f01035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.686+0000 7f99f4972700 1 -- 172.21.15.17:0/2362085565 >> 172.21.15.17:0/2362085565 conn(0x7f99f00780a0 msgr2=0x7f99f0105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.686+0000 7f99f4972700 1 -- 172.21.15.17:0/2362085565 shutdown_connections 2024-03-24T02:59:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:09.686+0000 7f99f4972700 1 -- 172.21.15.17:0/2362085565 wait complete. 2024-03-24T02:59:10.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:09 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2362085565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:59:10.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:09 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2362085565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:59:10.091 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:09 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2362085565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:59:10.511 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-24T02:59:10.511 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T02:59:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:11.007 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:10 smithi017 ceph-mon[31926]: pgmap v309: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 23 op/s 2024-03-24T02:59:11.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:10 smithi188 ceph-mon[36206]: pgmap v309: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 23 op/s 2024-03-24T02:59:11.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:10 smithi019 ceph-mon[35944]: pgmap v309: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 23 op/s 2024-03-24T02:59:12.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:12 smithi017 ceph-mon[31926]: pgmap v310: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-24T02:59:12.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:12 smithi188 ceph-mon[36206]: pgmap v310: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-24T02:59:12.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:12 smithi019 ceph-mon[35944]: pgmap v310: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-24T02:59:12.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.482+0000 7f967b995700 1 -- 172.21.15.17:0/112082223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f967406a930 msgr2=0x7f967406ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:12.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.482+0000 7f967b995700 1 --2- 172.21.15.17:0/112082223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f967406a930 0x7f967406ad10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f96700099c0 tx=0x7f9670009cd0 comp rx=0 tx=0).stop 2024-03-24T02:59:12.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.482+0000 7f967b995700 1 -- 172.21.15.17:0/112082223 shutdown_connections 2024-03-24T02:59:12.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.482+0000 7f967b995700 1 --2- 172.21.15.17:0/112082223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674107510 0x7f967410f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:12.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.482+0000 7f967b995700 1 --2- 172.21.15.17:0/112082223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f967406a930 0x7f967406ad10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:12.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.482+0000 7f967b995700 1 --2- 172.21.15.17:0/112082223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f967406b2e0 0x7f9674106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:12.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.482+0000 7f967b995700 1 -- 172.21.15.17:0/112082223 >> 172.21.15.17:0/112082223 conn(0x7f96740770e0 msgr2=0x7f96740774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:12.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.482+0000 7f967b995700 1 -- 172.21.15.17:0/112082223 shutdown_connections 2024-03-24T02:59:12.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.482+0000 7f967b995700 1 -- 172.21.15.17:0/112082223 wait complete. 2024-03-24T02:59:12.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f967b995700 1 Processor -- start 2024-03-24T02:59:12.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f967b995700 1 -- start start 2024-03-24T02:59:12.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f967b995700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f967406a930 0x7f9674101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f967b995700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f967406b2e0 0x7f96741016d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f967b995700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674107510 0x7f9674101c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f967b995700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96741124a0 con 0x7f967406b2e0 2024-03-24T02:59:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f967b995700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9674112320 con 0x7f9674107510 2024-03-24T02:59:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f967b995700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9674112620 con 0x7f967406a930 2024-03-24T02:59:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f9678f30700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f967406b2e0 0x7f96741016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f9679731700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f967406a930 0x7f9674101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f9678f30700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f967406b2e0 0x7f96741016d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42090/0 (socket says 172.21.15.17:42090) 2024-03-24T02:59:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f9678f30700 1 -- 172.21.15.17:0/3101447324 learned_addr learned my addr 172.21.15.17:0/3101447324 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.483+0000 7f9679f32700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674107510 0x7f9674101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.484+0000 7f9679731700 1 -- 172.21.15.17:0/3101447324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674107510 msgr2=0x7f9674101c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.484+0000 7f9679731700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674107510 0x7f9674101c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.484+0000 7f9679731700 1 -- 172.21.15.17:0/3101447324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f967406b2e0 msgr2=0x7f96741016d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.484+0000 7f9679731700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f967406b2e0 0x7f96741016d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.484+0000 7f9679731700 1 -- 172.21.15.17:0/3101447324 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96741a6eb0 con 0x7f967406a930 2024-03-24T02:59:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.484+0000 7f9679f32700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674107510 0x7f9674101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.484+0000 7f9679731700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f967406a930 0x7f9674101190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f96700098e0 tx=0x7f9670005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.484+0000 7f966a7fc700 1 -- 172.21.15.17:0/3101447324 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670027070 con 0x7f967406a930 2024-03-24T02:59:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.484+0000 7f967b995700 1 -- 172.21.15.17:0/3101447324 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96741a7140 con 0x7f967406a930 2024-03-24T02:59:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.484+0000 7f966a7fc700 1 -- 172.21.15.17:0/3101447324 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9670003680 con 0x7f967406a930 2024-03-24T02:59:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.484+0000 7f966a7fc700 1 -- 172.21.15.17:0/3101447324 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670022e10 con 0x7f967406a930 2024-03-24T02:59:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.484+0000 7f967b995700 1 -- 172.21.15.17:0/3101447324 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96741a75f0 con 0x7f967406a930 2024-03-24T02:59:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.485+0000 7f967b995700 1 -- 172.21.15.17:0/3101447324 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967410a6c0 con 0x7f967406a930 2024-03-24T02:59:12.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.487+0000 7f966a7fc700 1 -- 172.21.15.17:0/3101447324 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f967002a070 con 0x7f967406a930 2024-03-24T02:59:12.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.487+0000 7f966a7fc700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96600a2920 0x7f96600a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:12.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.487+0000 7f966a7fc700 1 -- 172.21.15.17:0/3101447324 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f96700cf820 con 0x7f967406a930 2024-03-24T02:59:12.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.487+0000 7f9678f30700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96600a2920 0x7f96600a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:12.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.488+0000 7f9678f30700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96600a2920 0x7f96600a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9674100c10 tx=0x7f9664009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:12.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.495+0000 7f966a7fc700 1 -- 172.21.15.17:0/3101447324 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9670013070 con 0x7f967406a930 2024-03-24T02:59:12.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.702+0000 7f967b995700 1 -- 172.21.15.17:0/3101447324 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f967410a0a0 con 0x7f967406a930 2024-03-24T02:59:12.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.705+0000 7f966a7fc700 1 -- 172.21.15.17:0/3101447324 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24100 (secure 0 0 0) 0x7f967009c550 con 0x7f967406a930 2024-03-24T02:59:12.705 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T02:59:12.706 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:58:41.346962+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T02:59:12.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.707+0000 7f967b995700 1 -- 172.21.15.17:0/3101447324 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96600a2920 msgr2=0x7f96600a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:12.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.707+0000 7f967b995700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96600a2920 0x7f96600a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9674100c10 tx=0x7f9664009260 comp rx=0 tx=0).stop 2024-03-24T02:59:12.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.707+0000 7f967b995700 1 -- 172.21.15.17:0/3101447324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f967406a930 msgr2=0x7f9674101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:12.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.707+0000 7f967b995700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f967406a930 0x7f9674101190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f96700098e0 tx=0x7f9670005f10 comp rx=0 tx=0).stop 2024-03-24T02:59:12.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.707+0000 7f967b995700 1 -- 172.21.15.17:0/3101447324 shutdown_connections 2024-03-24T02:59:12.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.707+0000 7f967b995700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96600a2920 0x7f96600a4de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:12.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.707+0000 7f967b995700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674107510 0x7f9674101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:12.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.707+0000 7f967b995700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f967406a930 0x7f9674101190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:12.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.707+0000 7f967b995700 1 --2- 172.21.15.17:0/3101447324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f967406b2e0 0x7f96741016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:12.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.707+0000 7f967b995700 1 -- 172.21.15.17:0/3101447324 >> 172.21.15.17:0/3101447324 conn(0x7f96740770e0 msgr2=0x7f9674108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:12.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.707+0000 7f967b995700 1 -- 172.21.15.17:0/3101447324 shutdown_connections 2024-03-24T02:59:12.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:12.707+0000 7f967b995700 1 -- 172.21.15.17:0/3101447324 wait complete. 2024-03-24T02:59:13.234 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:13 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3101447324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:59:13.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:13 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3101447324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:59:13.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:13 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3101447324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T02:59:13.507 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-24T02:59:13.507 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-24T02:59:13.508 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-24T02:59:13.508 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-24T02:59:13.508 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-24T02:59:13.509 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-24T02:59:13.509 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-24T02:59:13.509 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-24T02:59:13.510 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-24T02:59:13.510 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-24T02:59:13.511 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-24T02:59:13.511 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-24T02:59:14.152 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:14.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:14 smithi188 ceph-mon[36206]: pgmap v311: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 13 op/s 2024-03-24T02:59:14.313 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:14.314 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:14.314 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:14.314 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:14.315 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:14.317 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:14 smithi017 ceph-mon[31926]: pgmap v311: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 13 op/s 2024-03-24T02:59:14.318 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:14.318 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:14.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:14 smithi019 ceph-mon[35944]: pgmap v311: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 13 op/s 2024-03-24T02:59:14.554 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:14.555 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:14.555 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:14.558 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:16.096 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:16 smithi017 ceph-mon[31926]: pgmap v312: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-24T02:59:16.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:16 smithi188 ceph-mon[36206]: pgmap v312: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-24T02:59:16.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:16 smithi019 ceph-mon[35944]: pgmap v312: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-24T02:59:18.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:18 smithi188 ceph-mon[36206]: pgmap v313: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-24T02:59:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:18 smithi019 ceph-mon[35944]: pgmap v313: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-24T02:59:18.433 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:18 smithi017 ceph-mon[31926]: pgmap v313: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-24T02:59:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:20 smithi017 ceph-mon[31926]: pgmap v314: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-24T02:59:20.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:20 smithi188 ceph-mon[36206]: pgmap v314: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-24T02:59:20.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:20 smithi019 ceph-mon[35944]: pgmap v314: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-24T02:59:21.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.102+0000 7efe71170700 1 -- 172.21.15.17:0/2241598136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe6c1070d0 msgr2=0x7efe6c1074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:21.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.102+0000 7efe71170700 1 --2- 172.21.15.17:0/2241598136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe6c1070d0 0x7efe6c1074b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efe60009990 tx=0x7efe60009ca0 comp rx=0 tx=0).stop 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.102+0000 7efe71170700 1 -- 172.21.15.17:0/2241598136 shutdown_connections 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.102+0000 7efe71170700 1 --2- 172.21.15.17:0/2241598136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c101a30 0x7efe6c10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.102+0000 7efe71170700 1 --2- 172.21.15.17:0/2241598136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe6c1070d0 0x7efe6c1074b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.102+0000 7efe71170700 1 --2- 172.21.15.17:0/2241598136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe6c101090 0x7efe6c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.102+0000 7efe71170700 1 -- 172.21.15.17:0/2241598136 >> 172.21.15.17:0/2241598136 conn(0x7efe6c077fc0 msgr2=0x7efe6c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.102+0000 7efe71170700 1 -- 172.21.15.17:0/2241598136 shutdown_connections 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.102+0000 7efe71170700 1 -- 172.21.15.17:0/2241598136 wait complete. 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.103+0000 7efe71170700 1 Processor -- start 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.103+0000 7efe71170700 1 -- start start 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.103+0000 7efe71170700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c101090 0x7efe6c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.103+0000 7efe71170700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe6c101a30 0x7efe6c199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.103+0000 7efe71170700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe6c1070d0 0x7efe6c1a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.103+0000 7efe71170700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c111310 con 0x7efe6c101a30 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.103+0000 7efe71170700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c111190 con 0x7efe6c101090 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.103+0000 7efe71170700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c111490 con 0x7efe6c1070d0 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.103+0000 7efe6a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe6c101a30 0x7efe6c199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.103+0000 7efe6a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe6c101a30 0x7efe6c199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49228/0 (socket says 172.21.15.17:49228) 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.103+0000 7efe6a59c700 1 -- 172.21.15.17:0/3161970749 learned_addr learned my addr 172.21.15.17:0/3161970749 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.103+0000 7efe6ad9d700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c101090 0x7efe6c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe6b59e700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe6c1070d0 0x7efe6c1a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe6a59c700 1 -- 172.21.15.17:0/3161970749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe6c1070d0 msgr2=0x7efe6c1a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe6a59c700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe6c1070d0 0x7efe6c1a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe6a59c700 1 -- 172.21.15.17:0/3161970749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c101090 msgr2=0x7efe6c199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe6a59c700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c101090 0x7efe6c199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe6a59c700 1 -- 172.21.15.17:0/3161970749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c1a4cd0 con 0x7efe6c101a30 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe6ad9d700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c101090 0x7efe6c199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe6a59c700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe6c101a30 0x7efe6c199c50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00e9a0 tx=0x7efe5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe53fff700 1 -- 172.21.15.17:0/3161970749 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c00ca10 con 0x7efe6c101a30 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe53fff700 1 -- 172.21.15.17:0/3161970749 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe5c00cbb0 con 0x7efe6c101a30 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe71170700 1 -- 172.21.15.17:0/3161970749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c1a4f00 con 0x7efe6c101a30 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe53fff700 1 -- 172.21.15.17:0/3161970749 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c018680 con 0x7efe6c101a30 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.104+0000 7efe71170700 1 -- 172.21.15.17:0/3161970749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c1a53b0 con 0x7efe6c101a30 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.105+0000 7efe71170700 1 -- 172.21.15.17:0/3161970749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efe6c0628a0 con 0x7efe6c101a30 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.106+0000 7efe53fff700 1 -- 172.21.15.17:0/3161970749 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7efe5c010430 con 0x7efe6c101a30 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.107+0000 7efe53fff700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7efe540a2920 0x7efe540a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.107+0000 7efe53fff700 1 -- 172.21.15.17:0/3161970749 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efe5c0cf840 con 0x7efe6c101a30 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.107+0000 7efe6ad9d700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7efe540a2920 0x7efe540a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.107+0000 7efe53fff700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713] conn(0x7efe540ad5e0 0x7efe540afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.107+0000 7efe53fff700 1 -- 172.21.15.17:0/3161970749 --> [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe540b00f0 con 0x7efe540ad5e0 2024-03-24T02:59:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.107+0000 7efe53fff700 1 -- 172.21.15.17:0/3161970749 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe5c0cfc90 con 0x7efe6c101a30 2024-03-24T02:59:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.107+0000 7efe6b59e700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713] conn(0x7efe540ad5e0 0x7efe540afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.107+0000 7efe6ad9d700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7efe540a2920 0x7efe540a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efe6000b2e0 tx=0x7efe60005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.108+0000 7efe6b59e700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713] conn(0x7efe540ad5e0 0x7efe540afa20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.109+0000 7efe53fff700 1 -- 172.21.15.17:0/3161970749 <== osd.11 v2:172.21.15.188:6824/2579389713 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efe540b00f0 con 0x7efe540ad5e0 2024-03-24T02:59:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.126+0000 7efe71170700 1 -- 172.21.15.17:0/3161970749 --> [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efe6c062d20 con 0x7efe540ad5e0 2024-03-24T02:59:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.126+0000 7efe53fff700 1 -- 172.21.15.17:0/3161970749 <== osd.11 v2:172.21.15.188:6824/2579389713 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efe6c062d20 con 0x7efe540ad5e0 2024-03-24T02:59:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.127+0000 7efe71170700 1 -- 172.21.15.17:0/3161970749 >> [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713] conn(0x7efe540ad5e0 msgr2=0x7efe540afa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.127+0000 7efe71170700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713] conn(0x7efe540ad5e0 0x7efe540afa20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.127+0000 7efe71170700 1 -- 172.21.15.17:0/3161970749 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7efe540a2920 msgr2=0x7efe540a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.127+0000 7efe71170700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7efe540a2920 0x7efe540a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efe6000b2e0 tx=0x7efe60005960 comp rx=0 tx=0).stop 2024-03-24T02:59:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.127+0000 7efe71170700 1 -- 172.21.15.17:0/3161970749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe6c101a30 msgr2=0x7efe6c199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.127+0000 7efe71170700 1 --2- 172.21.15.17:0/3161970749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe6c101a30 0x7efe6c199c50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00e9a0 tx=0x7efe5c00ecb0 comp rx=0 tx=0).stop 2024-03-24T02:59:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.127+0000 7efe6b59e700 1 -- 172.21.15.17:0/3161970749 reap_dead start 2024-03-24T02:59:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.127+0000 7efe71170700 1 -- 172.21.15.17:0/3161970749 shutdown_connections 2024-03-24T02:59:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.127+0000 7efe71170700 1 -- 172.21.15.17:0/3161970749 >> 172.21.15.17:0/3161970749 conn(0x7efe6c077fc0 msgr2=0x7efe6c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.127+0000 7efe71170700 1 -- 172.21.15.17:0/3161970749 shutdown_connections 2024-03-24T02:59:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:21.127+0000 7efe71170700 1 -- 172.21.15.17:0/3161970749 wait complete. 2024-03-24T02:59:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:22 smithi017 ceph-mon[31926]: pgmap v315: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-24T02:59:22.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:22 smithi188 ceph-mon[36206]: pgmap v315: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-24T02:59:22.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:22 smithi019 ceph-mon[35944]: pgmap v315: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-24T02:59:23.548 INFO:teuthology.orchestra.run.smithi017.stdout:343597383703 2024-03-24T02:59:23.548 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-24T02:59:24.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:24 smithi188 ceph-mon[36206]: pgmap v316: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-24T02:59:24.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:24 smithi019 ceph-mon[35944]: pgmap v316: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-24T02:59:24.351 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:24 smithi017 ceph-mon[31926]: pgmap v316: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-24T02:59:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.836+0000 7f878ac54700 1 -- 172.21.15.17:0/1142248001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8784101370 msgr2=0x7f87841017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.836+0000 7f878ac54700 1 --2- 172.21.15.17:0/1142248001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8784101370 0x7f87841017f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8778009990 tx=0x7f8778009ca0 comp rx=0 tx=0).stop 2024-03-24T02:59:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.836+0000 7f878ac54700 1 -- 172.21.15.17:0/1142248001 shutdown_connections 2024-03-24T02:59:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.836+0000 7f878ac54700 1 --2- 172.21.15.17:0/1142248001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8784101d30 0x7f878410df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.836+0000 7f878ac54700 1 --2- 172.21.15.17:0/1142248001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8784107280 0x7f8784107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.836+0000 7f878ac54700 1 --2- 172.21.15.17:0/1142248001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8784101370 0x7f87841017f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:24.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.836+0000 7f878ac54700 1 -- 172.21.15.17:0/1142248001 >> 172.21.15.17:0/1142248001 conn(0x7f8784077ef0 msgr2=0x7f8784078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.836+0000 7f878ac54700 1 -- 172.21.15.17:0/1142248001 shutdown_connections 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.836+0000 7f878ac54700 1 -- 172.21.15.17:0/1142248001 wait complete. 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f878ac54700 1 Processor -- start 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f878ac54700 1 -- start start 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f878ac54700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8784101370 0x7f87841a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f878ac54700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8784101d30 0x7f87841a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f878ac54700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8784107280 0x7f878419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f878ac54700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8784111400 con 0x7f8784107280 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f878ac54700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8784111280 con 0x7f8784101d30 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f878ac54700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8784111580 con 0x7f8784101370 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f87891f1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8784107280 0x7f878419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f87889f0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8784101370 0x7f87841a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:24.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f87891f1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8784107280 0x7f878419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49244/0 (socket says 172.21.15.17:49244) 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f87891f1700 1 -- 172.21.15.17:0/571626683 learned_addr learned my addr 172.21.15.17:0/571626683 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f8783fff700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8784101d30 0x7f87841a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f87891f1700 1 -- 172.21.15.17:0/571626683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8784101370 msgr2=0x7f87841a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.837+0000 7f87891f1700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8784101370 0x7f87841a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.838+0000 7f87891f1700 1 -- 172.21.15.17:0/571626683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8784101d30 msgr2=0x7f87841a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.838+0000 7f87891f1700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8784101d30 0x7f87841a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.838+0000 7f87891f1700 1 -- 172.21.15.17:0/571626683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878419b110 con 0x7f8784107280 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.838+0000 7f87889f0700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8784101370 0x7f87841a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.838+0000 7f87891f1700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8784107280 0x7f878419a9b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f877c00efc0 tx=0x7f877c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.838+0000 7f8781ffb700 1 -- 172.21.15.17:0/571626683 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c0092e0 con 0x7f8784107280 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.838+0000 7f8781ffb700 1 -- 172.21.15.17:0/571626683 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f877c004bb0 con 0x7f8784107280 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.838+0000 7f878ac54700 1 -- 172.21.15.17:0/571626683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878419b3a0 con 0x7f8784107280 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.838+0000 7f8781ffb700 1 -- 172.21.15.17:0/571626683 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c004020 con 0x7f8784107280 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.838+0000 7f878ac54700 1 -- 172.21.15.17:0/571626683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87841a76c0 con 0x7f8784107280 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.839+0000 7f878ac54700 1 -- 172.21.15.17:0/571626683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f878406b2d0 con 0x7f8784107280 2024-03-24T02:59:24.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.840+0000 7f8781ffb700 1 -- 172.21.15.17:0/571626683 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f877c01b030 con 0x7f8784107280 2024-03-24T02:59:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.840+0000 7f8781ffb700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f876c0a29a0 0x7f876c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.841+0000 7f87889f0700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f876c0a29a0 0x7f876c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.841+0000 7f8781ffb700 1 -- 172.21.15.17:0/571626683 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f877c0ce6f0 con 0x7f8784107280 2024-03-24T02:59:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.841+0000 7f8781ffb700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962] conn(0x7f876c0ad5e0 0x7f876c0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.841+0000 7f8781ffb700 1 -- 172.21.15.17:0/571626683 --> [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f876c0b00f0 con 0x7f876c0ad5e0 2024-03-24T02:59:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.841+0000 7f8781ffb700 1 -- 172.21.15.17:0/571626683 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f877c0d5050 con 0x7f8784107280 2024-03-24T02:59:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.841+0000 7f8783fff700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962] conn(0x7f876c0ad5e0 0x7f876c0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.841+0000 7f87889f0700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f876c0a29a0 0x7f876c0a4e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f878419baf0 tx=0x7f8774009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.841+0000 7f8783fff700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962] conn(0x7f876c0ad5e0 0x7f876c0afa20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.842+0000 7f8781ffb700 1 -- 172.21.15.17:0/571626683 <== osd.4 v2:172.21.15.19:6808/4029793962 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f876c0b00f0 con 0x7f876c0ad5e0 2024-03-24T02:59:24.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.858+0000 7f878ac54700 1 -- 172.21.15.17:0/571626683 --> [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8784102810 con 0x7f876c0ad5e0 2024-03-24T02:59:24.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.859+0000 7f8781ffb700 1 -- 172.21.15.17:0/571626683 <== osd.4 v2:172.21.15.19:6808/4029793962 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8784102810 con 0x7f876c0ad5e0 2024-03-24T02:59:24.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.859+0000 7f878ac54700 1 -- 172.21.15.17:0/571626683 >> [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962] conn(0x7f876c0ad5e0 msgr2=0x7f876c0afa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:24.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.859+0000 7f878ac54700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962] conn(0x7f876c0ad5e0 0x7f876c0afa20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:24.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.859+0000 7f878ac54700 1 -- 172.21.15.17:0/571626683 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f876c0a29a0 msgr2=0x7f876c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:24.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.859+0000 7f878ac54700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f876c0a29a0 0x7f876c0a4e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f878419baf0 tx=0x7f8774009260 comp rx=0 tx=0).stop 2024-03-24T02:59:24.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.859+0000 7f878ac54700 1 -- 172.21.15.17:0/571626683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8784107280 msgr2=0x7f878419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:24.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.859+0000 7f878ac54700 1 --2- 172.21.15.17:0/571626683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8784107280 0x7f878419a9b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f877c00efc0 tx=0x7f877c00c3a0 comp rx=0 tx=0).stop 2024-03-24T02:59:24.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.859+0000 7f87891f1700 1 -- 172.21.15.17:0/571626683 reap_dead start 2024-03-24T02:59:24.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.859+0000 7f878ac54700 1 -- 172.21.15.17:0/571626683 shutdown_connections 2024-03-24T02:59:24.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.859+0000 7f878ac54700 1 -- 172.21.15.17:0/571626683 >> 172.21.15.17:0/571626683 conn(0x7f8784077ef0 msgr2=0x7f878406a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:24.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.859+0000 7f878ac54700 1 -- 172.21.15.17:0/571626683 shutdown_connections 2024-03-24T02:59:24.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:24.859+0000 7f878ac54700 1 -- 172.21.15.17:0/571626683 wait complete. 2024-03-24T02:59:26.023 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:26 smithi017 ceph-mon[31926]: pgmap v317: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-24T02:59:26.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:26 smithi188 ceph-mon[36206]: pgmap v317: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-24T02:59:26.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:26 smithi019 ceph-mon[35944]: pgmap v317: 129 pgs: 129 active+clean; 647 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-24T02:59:27.669 INFO:teuthology.orchestra.run.smithi017.stdout:141733920828 2024-03-24T02:59:27.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-24T02:59:28.283 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:28 smithi017 ceph-mon[31926]: pgmap v318: 129 pgs: 129 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-24T02:59:28.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:28 smithi188 ceph-mon[36206]: pgmap v318: 129 pgs: 129 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-24T02:59:28.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:28 smithi019 ceph-mon[35944]: pgmap v318: 129 pgs: 129 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-24T02:59:28.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.386+0000 7fad75128700 1 -- 172.21.15.17:0/819240168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad701074b0 msgr2=0x7fad7010f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:28.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.386+0000 7fad75128700 1 --2- 172.21.15.17:0/819240168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad701074b0 0x7fad7010f080 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fad6800b0d0 tx=0x7fad6800b3e0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.387+0000 7fad75128700 1 -- 172.21.15.17:0/819240168 shutdown_connections 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.387+0000 7fad75128700 1 --2- 172.21.15.17:0/819240168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad701074b0 0x7fad7010f080 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.387+0000 7fad75128700 1 --2- 172.21.15.17:0/819240168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad701023d0 0x7fad701027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.387+0000 7fad75128700 1 --2- 172.21.15.17:0/819240168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad70102d80 0x7fad70106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.387+0000 7fad75128700 1 -- 172.21.15.17:0/819240168 >> 172.21.15.17:0/819240168 conn(0x7fad700fdc40 msgr2=0x7fad70100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.387+0000 7fad75128700 1 -- 172.21.15.17:0/819240168 shutdown_connections 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.387+0000 7fad75128700 1 -- 172.21.15.17:0/819240168 wait complete. 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.387+0000 7fad75128700 1 Processor -- start 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad75128700 1 -- start start 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad75128700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad701023d0 0x7fad701a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad75128700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad70102d80 0x7fad701a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad75128700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad701074b0 0x7fad7019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad75128700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad701124c0 con 0x7fad701074b0 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad75128700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad70112340 con 0x7fad701023d0 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad75128700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad70112640 con 0x7fad70102d80 2024-03-24T02:59:28.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad6f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad701074b0 0x7fad7019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad6f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad701074b0 0x7fad7019a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49262/0 (socket says 172.21.15.17:49262) 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad6e59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad70102d80 0x7fad701a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad6f59e700 1 -- 172.21.15.17:0/3260478561 learned_addr learned my addr 172.21.15.17:0/3260478561 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad6ed9d700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad701023d0 0x7fad701a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad6f59e700 1 -- 172.21.15.17:0/3260478561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad70102d80 msgr2=0x7fad701a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad6f59e700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad70102d80 0x7fad701a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad6f59e700 1 -- 172.21.15.17:0/3260478561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad701023d0 msgr2=0x7fad701a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad6f59e700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad701023d0 0x7fad701a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad6f59e700 1 -- 172.21.15.17:0/3260478561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7019b100 con 0x7fad701074b0 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad6e59c700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad70102d80 0x7fad701a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.388+0000 7fad6f59e700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad701074b0 0x7fad7019a9a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fad68006070 tx=0x7fad68007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.389+0000 7fad57fff700 1 -- 172.21.15.17:0/3260478561 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68003c00 con 0x7fad701074b0 2024-03-24T02:59:28.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.389+0000 7fad75128700 1 -- 172.21.15.17:0/3260478561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7019b390 con 0x7fad701074b0 2024-03-24T02:59:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.389+0000 7fad57fff700 1 -- 172.21.15.17:0/3260478561 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad68022070 con 0x7fad701074b0 2024-03-24T02:59:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.389+0000 7fad57fff700 1 -- 172.21.15.17:0/3260478561 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6801b6c0 con 0x7fad701074b0 2024-03-24T02:59:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.389+0000 7fad75128700 1 -- 172.21.15.17:0/3260478561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad701a76b0 con 0x7fad701074b0 2024-03-24T02:59:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.389+0000 7fad75128700 1 -- 172.21.15.17:0/3260478561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad700ff840 con 0x7fad701074b0 2024-03-24T02:59:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.391+0000 7fad57fff700 1 -- 172.21.15.17:0/3260478561 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fad6801d470 con 0x7fad701074b0 2024-03-24T02:59:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.392+0000 7fad57fff700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fad580a2920 0x7fad580a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.393+0000 7fad6ed9d700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fad580a2920 0x7fad580a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.393+0000 7fad57fff700 1 -- 172.21.15.17:0/3260478561 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fad6800f020 con 0x7fad701074b0 2024-03-24T02:59:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.393+0000 7fad57fff700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482] conn(0x7fad580ad5d0 0x7fad580afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.393+0000 7fad57fff700 1 -- 172.21.15.17:0/3260478561 --> [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad68028e70 con 0x7fad580ad5d0 2024-03-24T02:59:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.393+0000 7fad57fff700 1 -- 172.21.15.17:0/3260478561 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad680d0020 con 0x7fad701074b0 2024-03-24T02:59:28.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.393+0000 7fad6ed9d700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fad580a2920 0x7fad580a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fad7019bae0 tx=0x7fad60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:28.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.393+0000 7fad6e59c700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482] conn(0x7fad580ad5d0 0x7fad580afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:28.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.394+0000 7fad6e59c700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482] conn(0x7fad580ad5d0 0x7fad580afa10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:28.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.395+0000 7fad57fff700 1 -- 172.21.15.17:0/3260478561 <== osd.2 v2:172.21.15.188:6800/2030207482 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fad68028e70 con 0x7fad580ad5d0 2024-03-24T02:59:28.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.412+0000 7fad75128700 1 -- 172.21.15.17:0/3260478561 --> [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad701038b0 con 0x7fad580ad5d0 2024-03-24T02:59:28.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.412+0000 7fad57fff700 1 -- 172.21.15.17:0/3260478561 <== osd.2 v2:172.21.15.188:6800/2030207482 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad701038b0 con 0x7fad580ad5d0 2024-03-24T02:59:28.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.413+0000 7fad75128700 1 -- 172.21.15.17:0/3260478561 >> [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482] conn(0x7fad580ad5d0 msgr2=0x7fad580afa10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:28.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.413+0000 7fad75128700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482] conn(0x7fad580ad5d0 0x7fad580afa10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.413+0000 7fad75128700 1 -- 172.21.15.17:0/3260478561 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fad580a2920 msgr2=0x7fad580a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:28.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.413+0000 7fad75128700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fad580a2920 0x7fad580a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fad7019bae0 tx=0x7fad60009260 comp rx=0 tx=0).stop 2024-03-24T02:59:28.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.413+0000 7fad75128700 1 -- 172.21.15.17:0/3260478561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad701074b0 msgr2=0x7fad7019a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:28.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.413+0000 7fad75128700 1 --2- 172.21.15.17:0/3260478561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad701074b0 0x7fad7019a9a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fad68006070 tx=0x7fad68007a20 comp rx=0 tx=0).stop 2024-03-24T02:59:28.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.413+0000 7fad6f59e700 1 -- 172.21.15.17:0/3260478561 reap_dead start 2024-03-24T02:59:28.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.413+0000 7fad75128700 1 -- 172.21.15.17:0/3260478561 shutdown_connections 2024-03-24T02:59:28.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.413+0000 7fad75128700 1 -- 172.21.15.17:0/3260478561 >> 172.21.15.17:0/3260478561 conn(0x7fad700fdc40 msgr2=0x7fad7010d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:28.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.413+0000 7fad75128700 1 -- 172.21.15.17:0/3260478561 shutdown_connections 2024-03-24T02:59:28.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.413+0000 7fad75128700 1 -- 172.21.15.17:0/3260478561 wait complete. 2024-03-24T02:59:28.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.933+0000 7f2745863700 1 -- 172.21.15.17:0/2960699701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27401071c0 msgr2=0x7f27401075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:28.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.933+0000 7f2745863700 1 --2- 172.21.15.17:0/2960699701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27401071c0 0x7f27401075a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2738009920 tx=0x7f2738009c30 comp rx=0 tx=0).stop 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.933+0000 7f2745863700 1 -- 172.21.15.17:0/2960699701 shutdown_connections 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.933+0000 7f2745863700 1 --2- 172.21.15.17:0/2960699701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2740101d30 0x7f274010df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.933+0000 7f2745863700 1 --2- 172.21.15.17:0/2960699701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27401071c0 0x7f27401075a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.933+0000 7f2745863700 1 --2- 172.21.15.17:0/2960699701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2740101370 0x7f27401017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.933+0000 7f2745863700 1 -- 172.21.15.17:0/2960699701 >> 172.21.15.17:0/2960699701 conn(0x7f2740077f50 msgr2=0x7f2740078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.933+0000 7f2745863700 1 -- 172.21.15.17:0/2960699701 shutdown_connections 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.933+0000 7f2745863700 1 -- 172.21.15.17:0/2960699701 wait complete. 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.933+0000 7f2745863700 1 Processor -- start 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f2745863700 1 -- start start 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f2745863700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2740101370 0x7f27401997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f2745863700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2740101d30 0x7f2740199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f2745863700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27401071c0 0x7f27401a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:28.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f2745863700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27401113a0 con 0x7f2740101d30 2024-03-24T02:59:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f2745863700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2740111220 con 0x7f2740101370 2024-03-24T02:59:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f2745863700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2740111520 con 0x7f27401071c0 2024-03-24T02:59:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f273e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2740101d30 0x7f2740199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f273e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2740101d30 0x7f2740199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49276/0 (socket says 172.21.15.17:49276) 2024-03-24T02:59:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f273e7fc700 1 -- 172.21.15.17:0/922515333 learned_addr learned my addr 172.21.15.17:0/922515333 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f273effd700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2740101370 0x7f27401997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f273f7fe700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27401071c0 0x7f27401a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f273e7fc700 1 -- 172.21.15.17:0/922515333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27401071c0 msgr2=0x7f27401a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f273e7fc700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27401071c0 0x7f27401a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f273e7fc700 1 -- 172.21.15.17:0/922515333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2740101370 msgr2=0x7f27401997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f273e7fc700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2740101370 0x7f27401997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.934+0000 7f273e7fc700 1 -- 172.21.15.17:0/922515333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27401a4d60 con 0x7f2740101d30 2024-03-24T02:59:28.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.935+0000 7f273e7fc700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2740101d30 0x7f2740199ce0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f272800e9c0 tx=0x7f272800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:28.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.935+0000 7f2744861700 1 -- 172.21.15.17:0/922515333 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272800cbd0 con 0x7f2740101d30 2024-03-24T02:59:28.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.935+0000 7f2744861700 1 -- 172.21.15.17:0/922515333 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f272800ee60 con 0x7f2740101d30 2024-03-24T02:59:28.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.935+0000 7f2745863700 1 -- 172.21.15.17:0/922515333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27401a4f90 con 0x7f2740101d30 2024-03-24T02:59:28.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.935+0000 7f2744861700 1 -- 172.21.15.17:0/922515333 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728005630 con 0x7f2740101d30 2024-03-24T02:59:28.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.935+0000 7f2745863700 1 -- 172.21.15.17:0/922515333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27401a53f0 con 0x7f2740101d30 2024-03-24T02:59:28.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.936+0000 7f2745863700 1 -- 172.21.15.17:0/922515333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f27400628a0 con 0x7f2740101d30 2024-03-24T02:59:28.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.937+0000 7f2744861700 1 -- 172.21.15.17:0/922515333 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f2728004750 con 0x7f2740101d30 2024-03-24T02:59:28.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.937+0000 7f2744861700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f272c0a2920 0x7f272c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.938+0000 7f273effd700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f272c0a2920 0x7f272c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.938+0000 7f2744861700 1 -- 172.21.15.17:0/922515333 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f27280cf4f0 con 0x7f2740101d30 2024-03-24T02:59:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.938+0000 7f2744861700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070] conn(0x7f272c0ad5e0 0x7f272c0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.938+0000 7f2744861700 1 -- 172.21.15.17:0/922515333 --> [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2728004500 con 0x7f272c0ad5e0 2024-03-24T02:59:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.938+0000 7f2744861700 1 -- 172.21.15.17:0/922515333 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27280cf940 con 0x7f2740101d30 2024-03-24T02:59:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.938+0000 7f273f7fe700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070] conn(0x7f272c0ad5e0 0x7f272c0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.938+0000 7f273effd700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f272c0a2920 0x7f272c0a4de0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f273800c010 tx=0x7f2738019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.939+0000 7f273f7fe700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070] conn(0x7f272c0ad5e0 0x7f272c0afa20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.940+0000 7f2744861700 1 -- 172.21.15.17:0/922515333 <== osd.9 v2:172.21.15.17:6826/116991070 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2728004500 con 0x7f272c0ad5e0 2024-03-24T02:59:28.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.956+0000 7f2745863700 1 -- 172.21.15.17:0/922515333 --> [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2740062d50 con 0x7f272c0ad5e0 2024-03-24T02:59:28.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.957+0000 7f2744861700 1 -- 172.21.15.17:0/922515333 <== osd.9 v2:172.21.15.17:6826/116991070 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2740062d50 con 0x7f272c0ad5e0 2024-03-24T02:59:28.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.957+0000 7f2745863700 1 -- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070] conn(0x7f272c0ad5e0 msgr2=0x7f272c0afa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:28.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.957+0000 7f2745863700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070] conn(0x7f272c0ad5e0 0x7f272c0afa20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.957+0000 7f2745863700 1 -- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f272c0a2920 msgr2=0x7f272c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:28.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.957+0000 7f2745863700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f272c0a2920 0x7f272c0a4de0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f273800c010 tx=0x7f2738019040 comp rx=0 tx=0).stop 2024-03-24T02:59:28.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.957+0000 7f2745863700 1 -- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2740101d30 msgr2=0x7f2740199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:28.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.957+0000 7f2745863700 1 --2- 172.21.15.17:0/922515333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2740101d30 0x7f2740199ce0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f272800e9c0 tx=0x7f272800ecd0 comp rx=0 tx=0).stop 2024-03-24T02:59:28.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.957+0000 7f273f7fe700 1 -- 172.21.15.17:0/922515333 reap_dead start 2024-03-24T02:59:28.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.957+0000 7f2745863700 1 -- 172.21.15.17:0/922515333 shutdown_connections 2024-03-24T02:59:28.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.957+0000 7f2745863700 1 -- 172.21.15.17:0/922515333 >> 172.21.15.17:0/922515333 conn(0x7f2740077f50 msgr2=0x7f2740100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:28.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.957+0000 7f2745863700 1 -- 172.21.15.17:0/922515333 shutdown_connections 2024-03-24T02:59:28.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:28.957+0000 7f2745863700 1 -- 172.21.15.17:0/922515333 wait complete. 2024-03-24T02:59:30.102 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]: dispatch 2024-03-24T02:59:30.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-24T02:59:30.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-24T02:59:30.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-03-24T02:59:30.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-03-24T02:59:30.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.35", "id": [1, 9]}]: dispatch 2024-03-24T02:59:30.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 6]}]: dispatch 2024-03-24T02:59:30.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:29 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-24T02:59:30.103 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:29 smithi017 ceph-mon[31926]: pgmap v319: 129 pgs: 129 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-24T02:59:30.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]: dispatch 2024-03-24T02:59:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-24T02:59:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-24T02:59:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-03-24T02:59:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-03-24T02:59:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.35", "id": [1, 9]}]: dispatch 2024-03-24T02:59:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 6]}]: dispatch 2024-03-24T02:59:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:29 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-24T02:59:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:29 smithi188 ceph-mon[36206]: pgmap v319: 129 pgs: 129 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-24T02:59:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]: dispatch 2024-03-24T02:59:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-24T02:59:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-24T02:59:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-03-24T02:59:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-03-24T02:59:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.35", "id": [1, 9]}]: dispatch 2024-03-24T02:59:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 6]}]: dispatch 2024-03-24T02:59:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:29 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-24T02:59:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:29 smithi019 ceph-mon[35944]: pgmap v319: 129 pgs: 129 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-24T02:59:30.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.959+0000 7fd7d6595700 1 -- 172.21.15.17:0/3623532248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d0102470 msgr2=0x7fd7d0102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:30.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.959+0000 7fd7d6595700 1 --2- 172.21.15.17:0/3623532248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d0102470 0x7fd7d0102850 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009990 tx=0x7fd7c0009ca0 comp rx=0 tx=0).stop 2024-03-24T02:59:30.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.959+0000 7fd7d6595700 1 -- 172.21.15.17:0/3623532248 shutdown_connections 2024-03-24T02:59:30.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.959+0000 7fd7d6595700 1 --2- 172.21.15.17:0/3623532248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d01074a0 0x7fd7d010f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:30.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.959+0000 7fd7d6595700 1 --2- 172.21.15.17:0/3623532248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d0102470 0x7fd7d0102850 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:30.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.959+0000 7fd7d6595700 1 --2- 172.21.15.17:0/3623532248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d0102e20 0x7fd7d0106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.959+0000 7fd7d6595700 1 -- 172.21.15.17:0/3623532248 >> 172.21.15.17:0/3623532248 conn(0x7fd7d0079220 msgr2=0x7fd7d0079630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.959+0000 7fd7d6595700 1 -- 172.21.15.17:0/3623532248 shutdown_connections 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.959+0000 7fd7d6595700 1 -- 172.21.15.17:0/3623532248 wait complete. 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.960+0000 7fd7d6595700 1 Processor -- start 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.960+0000 7fd7d6595700 1 -- start start 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.960+0000 7fd7d6595700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d0102470 0x7fd7d019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.960+0000 7fd7d6595700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d0102e20 0x7fd7d019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.960+0000 7fd7d6595700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01074a0 0x7fd7d0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.960+0000 7fd7d6595700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0112490 con 0x7fd7d0102470 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.960+0000 7fd7d6595700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0112310 con 0x7fd7d01074a0 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.960+0000 7fd7d6595700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0112610 con 0x7fd7d0102e20 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d0102470 0x7fd7d019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:30.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d0102470 0x7fd7d019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32852/0 (socket says 172.21.15.17:32852) 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cffff700 1 -- 172.21.15.17:0/1432090657 learned_addr learned my addr 172.21.15.17:0/1432090657 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cf7fe700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d0102e20 0x7fd7d019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7d4b32700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01074a0 0x7fd7d0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cffff700 1 -- 172.21.15.17:0/1432090657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d0102e20 msgr2=0x7fd7d019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cffff700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d0102e20 0x7fd7d019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cffff700 1 -- 172.21.15.17:0/1432090657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01074a0 msgr2=0x7fd7d0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cffff700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01074a0 0x7fd7d0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cffff700 1 -- 172.21.15.17:0/1432090657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d0199010 con 0x7fd7d0102470 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cffff700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d0102470 0x7fd7d019e660 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0000c00 tx=0x7fd7c000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cd7fa700 1 -- 172.21.15.17:0/1432090657 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0027070 con 0x7fd7d0102470 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7d6595700 1 -- 172.21.15.17:0/1432090657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d01992a0 con 0x7fd7d0102470 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cd7fa700 1 -- 172.21.15.17:0/1432090657 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7c0003d70 con 0x7fd7d0102470 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7cd7fa700 1 -- 172.21.15.17:0/1432090657 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000f460 con 0x7fd7d0102470 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.961+0000 7fd7d6595700 1 -- 172.21.15.17:0/1432090657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d01a55e0 con 0x7fd7d0102470 2024-03-24T02:59:30.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.962+0000 7fd7d6595700 1 -- 172.21.15.17:0/1432090657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd7d00628a0 con 0x7fd7d0102470 2024-03-24T02:59:30.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.963+0000 7fd7cd7fa700 1 -- 172.21.15.17:0/1432090657 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fd7c002a070 con 0x7fd7d0102470 2024-03-24T02:59:30.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.964+0000 7fd7cd7fa700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd7b80a2920 0x7fd7b80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:30.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.964+0000 7fd7cd7fa700 1 -- 172.21.15.17:0/1432090657 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fd7c00d0300 con 0x7fd7d0102470 2024-03-24T02:59:30.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.964+0000 7fd7cd7fa700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360] conn(0x7fd7b80ad900 0x7fd7b80afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:30.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.964+0000 7fd7cd7fa700 1 -- 172.21.15.17:0/1432090657 --> [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7b80b0470 con 0x7fd7b80ad900 2024-03-24T02:59:30.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.964+0000 7fd7cf7fe700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd7b80a2920 0x7fd7b80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:30.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.964+0000 7fd7cd7fa700 1 -- 172.21.15.17:0/1432090657 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c00d0750 con 0x7fd7d0102470 2024-03-24T02:59:30.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.965+0000 7fd7d4b32700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360] conn(0x7fd7b80ad900 0x7fd7b80afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:30.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.965+0000 7fd7cf7fe700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd7b80a2920 0x7fd7b80a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0199ef0 tx=0x7fd7c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:30.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.965+0000 7fd7d4b32700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360] conn(0x7fd7b80ad900 0x7fd7b80afda0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:30.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.966+0000 7fd7cd7fa700 1 -- 172.21.15.17:0/1432090657 <== osd.5 v2:172.21.15.188:6808/2503049360 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd7b80b0470 con 0x7fd7b80ad900 2024-03-24T02:59:30.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.988+0000 7fd7d6595700 1 -- 172.21.15.17:0/1432090657 --> [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd7d0062cd0 con 0x7fd7b80ad900 2024-03-24T02:59:30.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.988+0000 7fd7cd7fa700 1 -- 172.21.15.17:0/1432090657 <== osd.5 v2:172.21.15.188:6808/2503049360 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd7d0062cd0 con 0x7fd7b80ad900 2024-03-24T02:59:30.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.989+0000 7fd7d6595700 1 -- 172.21.15.17:0/1432090657 >> [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360] conn(0x7fd7b80ad900 msgr2=0x7fd7b80afda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:30.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.989+0000 7fd7d6595700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360] conn(0x7fd7b80ad900 0x7fd7b80afda0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:30.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.989+0000 7fd7d6595700 1 -- 172.21.15.17:0/1432090657 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd7b80a2920 msgr2=0x7fd7b80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:30.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.989+0000 7fd7d6595700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd7b80a2920 0x7fd7b80a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0199ef0 tx=0x7fd7c4009260 comp rx=0 tx=0).stop 2024-03-24T02:59:30.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.989+0000 7fd7d6595700 1 -- 172.21.15.17:0/1432090657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d0102470 msgr2=0x7fd7d019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:30.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.989+0000 7fd7d6595700 1 --2- 172.21.15.17:0/1432090657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7d0102470 0x7fd7d019e660 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0000c00 tx=0x7fd7c000bab0 comp rx=0 tx=0).stop 2024-03-24T02:59:30.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.989+0000 7fd7d4b32700 1 -- 172.21.15.17:0/1432090657 reap_dead start 2024-03-24T02:59:30.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.989+0000 7fd7d6595700 1 -- 172.21.15.17:0/1432090657 shutdown_connections 2024-03-24T02:59:30.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.989+0000 7fd7d6595700 1 -- 172.21.15.17:0/1432090657 >> 172.21.15.17:0/1432090657 conn(0x7fd7d0079220 msgr2=0x7fd7d0101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:30.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.989+0000 7fd7d6595700 1 -- 172.21.15.17:0/1432090657 shutdown_connections 2024-03-24T02:59:30.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:30.989+0000 7fd7d6595700 1 -- 172.21.15.17:0/1432090657 wait complete. 2024-03-24T02:59:31.075 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]': finished 2024-03-24T02:59:31.075 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-24T02:59:31.075 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-24T02:59:31.076 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-03-24T02:59:31.076 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-03-24T02:59:31.076 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.35", "id": [1, 9]}]': finished 2024-03-24T02:59:31.076 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 6]}]': finished 2024-03-24T02:59:31.076 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-24T02:59:31.076 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:30 smithi017 ceph-mon[31926]: osdmap e90: 12 total, 12 up, 12 in 2024-03-24T02:59:31.076 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:30 smithi017 ceph-mon[31926]: osdmap e91: 12 total, 12 up, 12 in 2024-03-24T02:59:31.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.213+0000 7f6415a05700 1 -- 172.21.15.17:0/1255809883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6410101af0 msgr2=0x7f6410101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.213+0000 7f6415a05700 1 --2- 172.21.15.17:0/1255809883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6410101af0 0x7f6410101ed0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f640000b0d0 tx=0x7f640000b3e0 comp rx=0 tx=0).stop 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.213+0000 7f6415a05700 1 -- 172.21.15.17:0/1255809883 shutdown_connections 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.213+0000 7f6415a05700 1 --2- 172.21.15.17:0/1255809883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6410106410 0x7f641010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.213+0000 7f6415a05700 1 --2- 172.21.15.17:0/1255809883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6410101af0 0x7f6410101ed0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.213+0000 7f6415a05700 1 --2- 172.21.15.17:0/1255809883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64101024a0 0x7f6410105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.213+0000 7f6415a05700 1 -- 172.21.15.17:0/1255809883 >> 172.21.15.17:0/1255809883 conn(0x7f6410077690 msgr2=0x7f64100feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.213+0000 7f6415a05700 1 -- 172.21.15.17:0/1255809883 shutdown_connections 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.213+0000 7f6415a05700 1 -- 172.21.15.17:0/1255809883 wait complete. 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.213+0000 7f6415a05700 1 Processor -- start 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f6415a05700 1 -- start start 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f6415a05700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6410101af0 0x7f641010e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f6415a05700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64101024a0 0x7f641010ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f6415a05700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6410106410 0x7f641010f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f6415a05700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6410112550 con 0x7f64101024a0 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f6415a05700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64101123d0 con 0x7f6410101af0 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f6415a05700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64101126d0 con 0x7f6410106410 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f640e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64101024a0 0x7f641010ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f640e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64101024a0 0x7f641010ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32860/0 (socket says 172.21.15.17:32860) 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f640e7fc700 1 -- 172.21.15.17:0/32802776 learned_addr learned my addr 172.21.15.17:0/32802776 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f640f7fe700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6410106410 0x7f641010f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f640effd700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6410101af0 0x7f641010e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f640e7fc700 1 -- 172.21.15.17:0/32802776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6410106410 msgr2=0x7f641010f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f640e7fc700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6410106410 0x7f641010f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f640e7fc700 1 -- 172.21.15.17:0/32802776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6410101af0 msgr2=0x7f641010e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f640e7fc700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6410101af0 0x7f641010e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:31.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f640e7fc700 1 -- 172.21.15.17:0/32802776 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64101a6f50 con 0x7f64101024a0 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f640f7fe700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6410106410 0x7f641010f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.214+0000 7f640effd700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6410101af0 0x7f641010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.215+0000 7f640e7fc700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64101024a0 0x7f641010ef00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f63f800e970 tx=0x7f63f800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.215+0000 7f6414a03700 1 -- 172.21.15.17:0/32802776 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8004a30 con 0x7f64101024a0 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.215+0000 7f6415a05700 1 -- 172.21.15.17:0/32802776 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64101a7240 con 0x7f64101024a0 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.215+0000 7f6414a03700 1 -- 172.21.15.17:0/32802776 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63f800ccc0 con 0x7f64101024a0 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.215+0000 7f6414a03700 1 -- 172.21.15.17:0/32802776 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8005650 con 0x7f64101024a0 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.215+0000 7f6415a05700 1 -- 172.21.15.17:0/32802776 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64101a77d0 con 0x7f64101024a0 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.216+0000 7f6415a05700 1 -- 172.21.15.17:0/32802776 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64100628a0 con 0x7f64101024a0 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.217+0000 7f6414a03700 1 -- 172.21.15.17:0/32802776 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f63f8004bd0 con 0x7f64101024a0 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.218+0000 7f6414a03700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f63fc0a2920 0x7f63fc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.218+0000 7f640effd700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f63fc0a2920 0x7f63fc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.218+0000 7f6414a03700 1 -- 172.21.15.17:0/32802776 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f63f80cf590 con 0x7f64101024a0 2024-03-24T02:59:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.218+0000 7f6414a03700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113] conn(0x7f63fc0ad980 0x7f63fc0afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:31.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.218+0000 7f6414a03700 1 -- 172.21.15.17:0/32802776 --> [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f63f8015e80 con 0x7f63fc0ad980 2024-03-24T02:59:31.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.218+0000 7f6414a03700 1 -- 172.21.15.17:0/32802776 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63f80cf9e0 con 0x7f64101024a0 2024-03-24T02:59:31.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.218+0000 7f640f7fe700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113] conn(0x7f63fc0ad980 0x7f63fc0afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:31.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.218+0000 7f640effd700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f63fc0a2920 0x7f63fc0a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6400009fd0 tx=0x7f640000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:31.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.218+0000 7f640f7fe700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113] conn(0x7f63fc0ad980 0x7f63fc0afe20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:31.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.219+0000 7f6414a03700 1 -- 172.21.15.17:0/32802776 <== osd.3 v2:172.21.15.17:6810/1744816113 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f63f8015e80 con 0x7f63fc0ad980 2024-03-24T02:59:31.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.236+0000 7f6415a05700 1 -- 172.21.15.17:0/32802776 --> [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6410062ca0 con 0x7f63fc0ad980 2024-03-24T02:59:31.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.236+0000 7f6414a03700 1 -- 172.21.15.17:0/32802776 <== osd.3 v2:172.21.15.17:6810/1744816113 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6410062ca0 con 0x7f63fc0ad980 2024-03-24T02:59:31.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.237+0000 7f6415a05700 1 -- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113] conn(0x7f63fc0ad980 msgr2=0x7f63fc0afe20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:31.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.237+0000 7f6415a05700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113] conn(0x7f63fc0ad980 0x7f63fc0afe20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:31.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.237+0000 7f6415a05700 1 -- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f63fc0a2920 msgr2=0x7f63fc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:31.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.237+0000 7f6415a05700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f63fc0a2920 0x7f63fc0a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6400009fd0 tx=0x7f640000b040 comp rx=0 tx=0).stop 2024-03-24T02:59:31.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.237+0000 7f6415a05700 1 -- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64101024a0 msgr2=0x7f641010ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:31.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.237+0000 7f6415a05700 1 --2- 172.21.15.17:0/32802776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64101024a0 0x7f641010ef00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f63f800e970 tx=0x7f63f800ec80 comp rx=0 tx=0).stop 2024-03-24T02:59:31.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.237+0000 7f640f7fe700 1 -- 172.21.15.17:0/32802776 reap_dead start 2024-03-24T02:59:31.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.237+0000 7f6415a05700 1 -- 172.21.15.17:0/32802776 shutdown_connections 2024-03-24T02:59:31.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.237+0000 7f6415a05700 1 -- 172.21.15.17:0/32802776 >> 172.21.15.17:0/32802776 conn(0x7f6410077690 msgr2=0x7f64100feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:31.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.237+0000 7f6415a05700 1 -- 172.21.15.17:0/32802776 shutdown_connections 2024-03-24T02:59:31.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:31.237+0000 7f6415a05700 1 -- 172.21.15.17:0/32802776 wait complete. 2024-03-24T02:59:31.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]': finished 2024-03-24T02:59:31.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-24T02:59:31.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-24T02:59:31.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-03-24T02:59:31.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-03-24T02:59:31.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.35", "id": [1, 9]}]': finished 2024-03-24T02:59:31.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 6]}]': finished 2024-03-24T02:59:31.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-24T02:59:31.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:30 smithi188 ceph-mon[36206]: osdmap e90: 12 total, 12 up, 12 in 2024-03-24T02:59:31.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:30 smithi188 ceph-mon[36206]: osdmap e91: 12 total, 12 up, 12 in 2024-03-24T02:59:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]': finished 2024-03-24T02:59:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-24T02:59:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-24T02:59:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-03-24T02:59:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-03-24T02:59:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.35", "id": [1, 9]}]': finished 2024-03-24T02:59:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 6]}]': finished 2024-03-24T02:59:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-24T02:59:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:30 smithi019 ceph-mon[35944]: osdmap e90: 12 total, 12 up, 12 in 2024-03-24T02:59:31.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:30 smithi019 ceph-mon[35944]: osdmap e91: 12 total, 12 up, 12 in 2024-03-24T02:59:32.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:31 smithi017 ceph-mon[31926]: pgmap v322: 129 pgs: 129 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-24T02:59:32.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:31 smithi188 ceph-mon[36206]: pgmap v322: 129 pgs: 129 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-24T02:59:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:31 smithi019 ceph-mon[35944]: pgmap v322: 129 pgs: 129 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-24T02:59:32.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.664+0000 7fe44c955700 1 -- 172.21.15.17:0/4194058828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4481081c0 msgr2=0x7fe4481085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.664+0000 7fe44c955700 1 --2- 172.21.15.17:0/4194058828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4481081c0 0x7fe4481085a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe4380099c0 tx=0x7fe438009cd0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.664+0000 7fe44c955700 1 -- 172.21.15.17:0/4194058828 shutdown_connections 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.664+0000 7fe44c955700 1 --2- 172.21.15.17:0/4194058828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe448102ba0 0x7fe44810efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.664+0000 7fe44c955700 1 --2- 172.21.15.17:0/4194058828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4481081c0 0x7fe4481085a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.664+0000 7fe44c955700 1 --2- 172.21.15.17:0/4194058828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4481021e0 0x7fe448102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.664+0000 7fe44c955700 1 -- 172.21.15.17:0/4194058828 >> 172.21.15.17:0/4194058828 conn(0x7fe4480792c0 msgr2=0x7fe4480796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.664+0000 7fe44c955700 1 -- 172.21.15.17:0/4194058828 shutdown_connections 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.664+0000 7fe44c955700 1 -- 172.21.15.17:0/4194058828 wait complete. 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.665+0000 7fe44c955700 1 Processor -- start 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.665+0000 7fe44c955700 1 -- start start 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.665+0000 7fe44c955700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4481021e0 0x7fe44819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.665+0000 7fe44c955700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe448102ba0 0x7fe44819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.665+0000 7fe44c955700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4481081c0 0x7fe448198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.665+0000 7fe44c955700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe448112470 con 0x7fe4481021e0 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.665+0000 7fe44c955700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4481122f0 con 0x7fe4481081c0 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.665+0000 7fe44c955700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4481125f0 con 0x7fe448102ba0 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.665+0000 7fe44659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4481021e0 0x7fe44819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.665+0000 7fe44659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4481021e0 0x7fe44819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32884/0 (socket says 172.21.15.17:32884) 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.665+0000 7fe44659c700 1 -- 172.21.15.17:0/2309709497 learned_addr learned my addr 172.21.15.17:0/2309709497 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.665+0000 7fe445d9b700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe448102ba0 0x7fe44819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.666+0000 7fe446d9d700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4481081c0 0x7fe448198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.666+0000 7fe44659c700 1 -- 172.21.15.17:0/2309709497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe448102ba0 msgr2=0x7fe44819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.666+0000 7fe44659c700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe448102ba0 0x7fe44819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.666+0000 7fe44659c700 1 -- 172.21.15.17:0/2309709497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4481081c0 msgr2=0x7fe448198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.666+0000 7fe44659c700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4481081c0 0x7fe448198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.666+0000 7fe44659c700 1 -- 172.21.15.17:0/2309709497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe448199030 con 0x7fe4481021e0 2024-03-24T02:59:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.666+0000 7fe44659c700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4481021e0 0x7fe44819e680 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe438006010 tx=0x7fe43800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:32.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.666+0000 7fe4377fe700 1 -- 172.21.15.17:0/2309709497 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438003680 con 0x7fe4481021e0 2024-03-24T02:59:32.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.666+0000 7fe44c955700 1 -- 172.21.15.17:0/2309709497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4481992c0 con 0x7fe4481021e0 2024-03-24T02:59:32.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.666+0000 7fe4377fe700 1 -- 172.21.15.17:0/2309709497 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4380050d0 con 0x7fe4481021e0 2024-03-24T02:59:32.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.666+0000 7fe4377fe700 1 -- 172.21.15.17:0/2309709497 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438003db0 con 0x7fe4481021e0 2024-03-24T02:59:32.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.666+0000 7fe44c955700 1 -- 172.21.15.17:0/2309709497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4481a5430 con 0x7fe4481021e0 2024-03-24T02:59:32.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.667+0000 7fe44c955700 1 -- 172.21.15.17:0/2309709497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe4480628a0 con 0x7fe4481021e0 2024-03-24T02:59:32.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.668+0000 7fe4377fe700 1 -- 172.21.15.17:0/2309709497 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fe438005280 con 0x7fe4481021e0 2024-03-24T02:59:32.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.669+0000 7fe4377fe700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe4300a2ac0 0x7fe4300a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:32.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.669+0000 7fe445d9b700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe4300a2ac0 0x7fe4300a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:32.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.669+0000 7fe4377fe700 1 -- 172.21.15.17:0/2309709497 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fe4380d0180 con 0x7fe4481021e0 2024-03-24T02:59:32.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.669+0000 7fe4377fe700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279] conn(0x7fe4300adb20 0x7fe4300affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:32.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.669+0000 7fe4377fe700 1 -- 172.21.15.17:0/2309709497 --> [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe43802ae50 con 0x7fe4300adb20 2024-03-24T02:59:32.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.669+0000 7fe4377fe700 1 -- 172.21.15.17:0/2309709497 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4380d05d0 con 0x7fe4481021e0 2024-03-24T02:59:32.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.669+0000 7fe446d9d700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279] conn(0x7fe4300adb20 0x7fe4300affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:32.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.669+0000 7fe445d9b700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe4300a2ac0 0x7fe4300a4f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe448199f10 tx=0x7fe43c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:32.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.670+0000 7fe446d9d700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279] conn(0x7fe4300adb20 0x7fe4300affc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:32.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.671+0000 7fe4377fe700 1 -- 172.21.15.17:0/2309709497 <== osd.8 v2:172.21.15.188:6816/1144752279 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe43802ae50 con 0x7fe4300adb20 2024-03-24T02:59:32.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.687+0000 7fe44c955700 1 -- 172.21.15.17:0/2309709497 --> [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe448062cd0 con 0x7fe4300adb20 2024-03-24T02:59:32.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.687+0000 7fe4377fe700 1 -- 172.21.15.17:0/2309709497 <== osd.8 v2:172.21.15.188:6816/1144752279 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe448062cd0 con 0x7fe4300adb20 2024-03-24T02:59:32.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.687+0000 7fe44c955700 1 -- 172.21.15.17:0/2309709497 >> [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279] conn(0x7fe4300adb20 msgr2=0x7fe4300affc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:32.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.687+0000 7fe44c955700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279] conn(0x7fe4300adb20 0x7fe4300affc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.687+0000 7fe44c955700 1 -- 172.21.15.17:0/2309709497 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe4300a2ac0 msgr2=0x7fe4300a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:32.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.687+0000 7fe44c955700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe4300a2ac0 0x7fe4300a4f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe448199f10 tx=0x7fe43c009260 comp rx=0 tx=0).stop 2024-03-24T02:59:32.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.687+0000 7fe44c955700 1 -- 172.21.15.17:0/2309709497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4481021e0 msgr2=0x7fe44819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:32.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.687+0000 7fe44c955700 1 --2- 172.21.15.17:0/2309709497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4481021e0 0x7fe44819e680 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe438006010 tx=0x7fe43800f690 comp rx=0 tx=0).stop 2024-03-24T02:59:32.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.687+0000 7fe446d9d700 1 -- 172.21.15.17:0/2309709497 reap_dead start 2024-03-24T02:59:32.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.688+0000 7fe44c955700 1 -- 172.21.15.17:0/2309709497 shutdown_connections 2024-03-24T02:59:32.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.688+0000 7fe44c955700 1 -- 172.21.15.17:0/2309709497 >> 172.21.15.17:0/2309709497 conn(0x7fe4480792c0 msgr2=0x7fe448100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:32.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.688+0000 7fe44c955700 1 -- 172.21.15.17:0/2309709497 shutdown_connections 2024-03-24T02:59:32.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.688+0000 7fe44c955700 1 -- 172.21.15.17:0/2309709497 wait complete. 2024-03-24T02:59:32.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.899+0000 7f25a56bd700 1 -- 172.21.15.17:0/4043093604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25a0069e40 msgr2=0x7f25a006a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:32.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.899+0000 7f25a56bd700 1 --2- 172.21.15.17:0/4043093604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25a0069e40 0x7f25a006a2a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2594009990 tx=0x7f2594009ca0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.899+0000 7f25a56bd700 1 -- 172.21.15.17:0/4043093604 shutdown_connections 2024-03-24T02:59:32.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.899+0000 7f25a56bd700 1 --2- 172.21.15.17:0/4043093604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a006a7e0 0x7f25a010f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.899+0000 7f25a56bd700 1 --2- 172.21.15.17:0/4043093604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25a01081e0 0x7f25a01085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.899+0000 7f25a56bd700 1 --2- 172.21.15.17:0/4043093604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25a0069e40 0x7f25a006a2a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.899+0000 7f25a56bd700 1 -- 172.21.15.17:0/4043093604 >> 172.21.15.17:0/4043093604 conn(0x7f25a0077fc0 msgr2=0x7f25a00783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:32.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.900+0000 7f25a56bd700 1 -- 172.21.15.17:0/4043093604 shutdown_connections 2024-03-24T02:59:32.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.900+0000 7f25a56bd700 1 -- 172.21.15.17:0/4043093604 wait complete. 2024-03-24T02:59:32.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.900+0000 7f25a56bd700 1 Processor -- start 2024-03-24T02:59:32.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.900+0000 7f25a56bd700 1 -- start start 2024-03-24T02:59:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.900+0000 7f25a56bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25a0069e40 0x7f25a0101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.900+0000 7f25a56bd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25a006a7e0 0x7f25a01016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.900+0000 7f25a56bd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a01081e0 0x7f25a0101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.900+0000 7f25a56bd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0112440 con 0x7f25a006a7e0 2024-03-24T02:59:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.900+0000 7f25a56bd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a01122c0 con 0x7f25a0069e40 2024-03-24T02:59:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.900+0000 7f25a56bd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a01125c0 con 0x7f25a01081e0 2024-03-24T02:59:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a01081e0 0x7f25a0101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a01081e0 0x7f25a0101be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:34566/0 (socket says 172.21.15.17:34566) 2024-03-24T02:59:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259f7fe700 1 -- 172.21.15.17:0/2631554226 learned_addr learned my addr 172.21.15.17:0/2631554226 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259effd700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25a0069e40 0x7f25a0101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259e7fc700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25a006a7e0 0x7f25a01016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259effd700 1 -- 172.21.15.17:0/2631554226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a01081e0 msgr2=0x7f25a0101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259effd700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a01081e0 0x7f25a0101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259effd700 1 -- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25a006a7e0 msgr2=0x7f25a01016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259effd700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25a006a7e0 0x7f25a01016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259effd700 1 -- 172.21.15.17:0/2631554226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a01a6e00 con 0x7f25a0069e40 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259f7fe700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a01081e0 0x7f25a0101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259e7fc700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25a006a7e0 0x7f25a01016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f259effd700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25a0069e40 0x7f25a0101160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f259000e9f0 tx=0x7f259000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f2587fff700 1 -- 172.21.15.17:0/2631554226 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259000cb70 con 0x7f25a0069e40 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f25a56bd700 1 -- 172.21.15.17:0/2631554226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a01a70f0 con 0x7f25a0069e40 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f2587fff700 1 -- 172.21.15.17:0/2631554226 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f259000ee90 con 0x7f25a0069e40 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f2587fff700 1 -- 172.21.15.17:0/2631554226 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590003dc0 con 0x7f25a0069e40 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.901+0000 7f25a56bd700 1 -- 172.21.15.17:0/2631554226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a01a7650 con 0x7f25a0069e40 2024-03-24T02:59:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.902+0000 7f25a56bd700 1 -- 172.21.15.17:0/2631554226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25a00628a0 con 0x7f25a0069e40 2024-03-24T02:59:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.904+0000 7f2587fff700 1 -- 172.21.15.17:0/2631554226 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f2590003f60 con 0x7f25a0069e40 2024-03-24T02:59:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.904+0000 7f2587fff700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25880a2920 0x7f25880a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.904+0000 7f259e7fc700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25880a2920 0x7f25880a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.904+0000 7f2587fff700 1 -- 172.21.15.17:0/2631554226 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f25900cf450 con 0x7f25a0069e40 2024-03-24T02:59:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.905+0000 7f2587fff700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551] conn(0x7f25880ad980 0x7f25880afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.905+0000 7f2587fff700 1 -- 172.21.15.17:0/2631554226 --> [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25900090b0 con 0x7f25880ad980 2024-03-24T02:59:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.905+0000 7f2587fff700 1 -- 172.21.15.17:0/2631554226 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25900cf800 con 0x7f25a0069e40 2024-03-24T02:59:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.905+0000 7f259f7fe700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551] conn(0x7f25880ad980 0x7f25880afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.905+0000 7f259e7fc700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25880a2920 0x7f25880a4de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f259400b2e0 tx=0x7f259401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.905+0000 7f259f7fe700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551] conn(0x7f25880ad980 0x7f25880afe20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.906+0000 7f2587fff700 1 -- 172.21.15.17:0/2631554226 <== osd.0 v2:172.21.15.17:6802/3184461551 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f25900090b0 con 0x7f25880ad980 2024-03-24T02:59:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.923+0000 7f25a56bd700 1 -- 172.21.15.17:0/2631554226 --> [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f25a0062ca0 con 0x7f25880ad980 2024-03-24T02:59:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.923+0000 7f2587fff700 1 -- 172.21.15.17:0/2631554226 <== osd.0 v2:172.21.15.17:6802/3184461551 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f25a0062ca0 con 0x7f25880ad980 2024-03-24T02:59:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.924+0000 7f25a56bd700 1 -- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551] conn(0x7f25880ad980 msgr2=0x7f25880afe20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.924+0000 7f25a56bd700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551] conn(0x7f25880ad980 0x7f25880afe20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.924+0000 7f25a56bd700 1 -- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25880a2920 msgr2=0x7f25880a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.924+0000 7f25a56bd700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25880a2920 0x7f25880a4de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f259400b2e0 tx=0x7f259401a040 comp rx=0 tx=0).stop 2024-03-24T02:59:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.924+0000 7f25a56bd700 1 -- 172.21.15.17:0/2631554226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25a0069e40 msgr2=0x7f25a0101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.924+0000 7f25a56bd700 1 --2- 172.21.15.17:0/2631554226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25a0069e40 0x7f25a0101160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f259000e9f0 tx=0x7f259000ed00 comp rx=0 tx=0).stop 2024-03-24T02:59:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.924+0000 7f259f7fe700 1 -- 172.21.15.17:0/2631554226 reap_dead start 2024-03-24T02:59:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.924+0000 7f25a56bd700 1 -- 172.21.15.17:0/2631554226 shutdown_connections 2024-03-24T02:59:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.924+0000 7f25a56bd700 1 -- 172.21.15.17:0/2631554226 >> 172.21.15.17:0/2631554226 conn(0x7f25a0077fc0 msgr2=0x7f25a00ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.924+0000 7f25a56bd700 1 -- 172.21.15.17:0/2631554226 shutdown_connections 2024-03-24T02:59:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:32.924+0000 7f25a56bd700 1 -- 172.21.15.17:0/2631554226 wait complete. 2024-03-24T02:59:33.352 INFO:teuthology.orchestra.run.smithi017.stdout:77309411398 2024-03-24T02:59:33.353 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-24T02:59:34.002 INFO:teuthology.orchestra.run.smithi017.stdout:253403070505 2024-03-24T02:59:34.003 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-24T02:59:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.211+0000 7fbe2763e700 1 -- 172.21.15.17:0/459405974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe20100780 msgr2=0x7fbe20100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.211+0000 7fbe2763e700 1 --2- 172.21.15.17:0/459405974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe20100780 0x7fbe20100c00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009990 tx=0x7fbe10009ca0 comp rx=0 tx=0).stop 2024-03-24T02:59:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.211+0000 7fbe2763e700 1 -- 172.21.15.17:0/459405974 shutdown_connections 2024-03-24T02:59:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.211+0000 7fbe2763e700 1 --2- 172.21.15.17:0/459405974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20101140 0x7fbe2010f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.211+0000 7fbe2763e700 1 --2- 172.21.15.17:0/459405974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe20107200 0x7fbe201075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.211+0000 7fbe2763e700 1 --2- 172.21.15.17:0/459405974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe20100780 0x7fbe20100c00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.211+0000 7fbe2763e700 1 -- 172.21.15.17:0/459405974 >> 172.21.15.17:0/459405974 conn(0x7fbe20077690 msgr2=0x7fbe200feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.211+0000 7fbe2763e700 1 -- 172.21.15.17:0/459405974 shutdown_connections 2024-03-24T02:59:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.211+0000 7fbe2763e700 1 -- 172.21.15.17:0/459405974 wait complete. 2024-03-24T02:59:34.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe2763e700 1 Processor -- start 2024-03-24T02:59:34.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe2763e700 1 -- start start 2024-03-24T02:59:34.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe2763e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe20100780 0x7fbe201a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:34.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe2763e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20101140 0x7fbe201a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:34.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe2763e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe20107200 0x7fbe2019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe2763e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe20112610 con 0x7fbe20100780 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe2763e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe20112490 con 0x7fbe20101140 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe2763e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe20112790 con 0x7fbe20107200 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe253da700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe20100780 0x7fbe201a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe24bd9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20101140 0x7fbe201a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe24bd9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20101140 0x7fbe201a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:58772/0 (socket says 172.21.15.17:58772) 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe253da700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe20100780 0x7fbe201a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32920/0 (socket says 172.21.15.17:32920) 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe24bd9700 1 -- 172.21.15.17:0/3058271303 learned_addr learned my addr 172.21.15.17:0/3058271303 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.212+0000 7fbe25bdb700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe20107200 0x7fbe2019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.213+0000 7fbe24bd9700 1 -- 172.21.15.17:0/3058271303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe20107200 msgr2=0x7fbe2019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.213+0000 7fbe24bd9700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe20107200 0x7fbe2019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.213+0000 7fbe24bd9700 1 -- 172.21.15.17:0/3058271303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe20100780 msgr2=0x7fbe201a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.213+0000 7fbe24bd9700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe20100780 0x7fbe201a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.213+0000 7fbe24bd9700 1 -- 172.21.15.17:0/3058271303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2019b1f0 con 0x7fbe20101140 2024-03-24T02:59:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.213+0000 7fbe24bd9700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20101140 0x7fbe201a0e10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe10005910 tx=0x7fbe1000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.213+0000 7fbe167fc700 1 -- 172.21.15.17:0/3058271303 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10027070 con 0x7fbe20101140 2024-03-24T02:59:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.213+0000 7fbe167fc700 1 -- 172.21.15.17:0/3058271303 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe10004650 con 0x7fbe20101140 2024-03-24T02:59:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.213+0000 7fbe167fc700 1 -- 172.21.15.17:0/3058271303 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000f460 con 0x7fbe20101140 2024-03-24T02:59:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.213+0000 7fbe2763e700 1 -- 172.21.15.17:0/3058271303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2019b480 con 0x7fbe20101140 2024-03-24T02:59:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.213+0000 7fbe2763e700 1 -- 172.21.15.17:0/3058271303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe201a77a0 con 0x7fbe20101140 2024-03-24T02:59:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.214+0000 7fbe2763e700 1 -- 172.21.15.17:0/3058271303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe20102b70 con 0x7fbe20101140 2024-03-24T02:59:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.215+0000 7fbe167fc700 1 -- 172.21.15.17:0/3058271303 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fbe1002a070 con 0x7fbe20101140 2024-03-24T02:59:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.216+0000 7fbe167fc700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbe0c0a2920 0x7fbe0c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:34.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.216+0000 7fbe167fc700 1 -- 172.21.15.17:0/3058271303 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fbe100cff90 con 0x7fbe20101140 2024-03-24T02:59:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.216+0000 7fbe253da700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbe0c0a2920 0x7fbe0c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.216+0000 7fbe167fc700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892] conn(0x7fbe0c0ad980 0x7fbe0c0afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.216+0000 7fbe167fc700 1 -- 172.21.15.17:0/3058271303 --> [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe10001d30 con 0x7fbe0c0ad980 2024-03-24T02:59:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.216+0000 7fbe167fc700 1 -- 172.21.15.17:0/3058271303 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe100d03e0 con 0x7fbe20101140 2024-03-24T02:59:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.216+0000 7fbe25bdb700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892] conn(0x7fbe0c0ad980 0x7fbe0c0afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.216+0000 7fbe253da700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbe0c0a2920 0x7fbe0c0a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe2019bbd0 tx=0x7fbe1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.217+0000 7fbe25bdb700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892] conn(0x7fbe0c0ad980 0x7fbe0c0afe20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:34.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.217+0000 7fbe167fc700 1 -- 172.21.15.17:0/3058271303 <== osd.10 v2:172.21.15.19:6824/3678221892 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbe10001d30 con 0x7fbe0c0ad980 2024-03-24T02:59:34.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.234+0000 7fbe2763e700 1 -- 172.21.15.17:0/3058271303 --> [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe20068e50 con 0x7fbe0c0ad980 2024-03-24T02:59:34.234 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:34 smithi017 ceph-mon[31926]: pgmap v323: 129 pgs: 129 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-24T02:59:34.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.234+0000 7fbe167fc700 1 -- 172.21.15.17:0/3058271303 <== osd.10 v2:172.21.15.19:6824/3678221892 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbe20068e50 con 0x7fbe0c0ad980 2024-03-24T02:59:34.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.235+0000 7fbe2763e700 1 -- 172.21.15.17:0/3058271303 >> [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892] conn(0x7fbe0c0ad980 msgr2=0x7fbe0c0afe20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.235+0000 7fbe2763e700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892] conn(0x7fbe0c0ad980 0x7fbe0c0afe20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.235+0000 7fbe2763e700 1 -- 172.21.15.17:0/3058271303 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbe0c0a2920 msgr2=0x7fbe0c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.235+0000 7fbe2763e700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbe0c0a2920 0x7fbe0c0a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe2019bbd0 tx=0x7fbe1c009260 comp rx=0 tx=0).stop 2024-03-24T02:59:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.235+0000 7fbe2763e700 1 -- 172.21.15.17:0/3058271303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20101140 msgr2=0x7fbe201a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.235+0000 7fbe2763e700 1 --2- 172.21.15.17:0/3058271303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe20101140 0x7fbe201a0e10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe10005910 tx=0x7fbe1000bab0 comp rx=0 tx=0).stop 2024-03-24T02:59:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.235+0000 7fbe25bdb700 1 -- 172.21.15.17:0/3058271303 reap_dead start 2024-03-24T02:59:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.235+0000 7fbe2763e700 1 -- 172.21.15.17:0/3058271303 shutdown_connections 2024-03-24T02:59:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.235+0000 7fbe2763e700 1 -- 172.21.15.17:0/3058271303 >> 172.21.15.17:0/3058271303 conn(0x7fbe20077690 msgr2=0x7fbe20105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.235+0000 7fbe2763e700 1 -- 172.21.15.17:0/3058271303 shutdown_connections 2024-03-24T02:59:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:34.235+0000 7fbe2763e700 1 -- 172.21.15.17:0/3058271303 wait complete. 2024-03-24T02:59:34.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:34 smithi188 ceph-mon[36206]: pgmap v323: 129 pgs: 129 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-24T02:59:34.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:34 smithi019 ceph-mon[35944]: pgmap v323: 129 pgs: 129 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-24T02:59:35.103 INFO:teuthology.orchestra.run.smithi017.stdout:163208757304 2024-03-24T02:59:35.103 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-24T02:59:35.413 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:35 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:59:35.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:35 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:59:35.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:35 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T02:59:36.200 INFO:teuthology.orchestra.run.smithi017.stdout:107374182465 2024-03-24T02:59:36.200 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-24T02:59:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:36 smithi019 ceph-mon[35944]: pgmap v324: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-24T02:59:36.429 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:36 smithi017 ceph-mon[31926]: pgmap v324: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-24T02:59:36.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:36 smithi188 ceph-mon[36206]: pgmap v324: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-24T02:59:36.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.878+0000 7ff61e6a5700 1 -- 172.21.15.17:0/1080052102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff618101350 msgr2=0x7ff618101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:36.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.878+0000 7ff61e6a5700 1 --2- 172.21.15.17:0/1080052102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff618101350 0x7ff618101730 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff6080099c0 tx=0x7ff608009cd0 comp rx=0 tx=0).stop 2024-03-24T02:59:36.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.878+0000 7ff61e6a5700 1 -- 172.21.15.17:0/1080052102 shutdown_connections 2024-03-24T02:59:36.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.878+0000 7ff61e6a5700 1 --2- 172.21.15.17:0/1080052102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6181063a0 0x7ff61810df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:36.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.878+0000 7ff61e6a5700 1 --2- 172.21.15.17:0/1080052102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff618101350 0x7ff618101730 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:36.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.878+0000 7ff61e6a5700 1 --2- 172.21.15.17:0/1080052102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618101d00 0x7ff618105cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:36.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.878+0000 7ff61e6a5700 1 -- 172.21.15.17:0/1080052102 >> 172.21.15.17:0/1080052102 conn(0x7ff6180770e0 msgr2=0x7ff6180774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:36.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.878+0000 7ff61e6a5700 1 -- 172.21.15.17:0/1080052102 shutdown_connections 2024-03-24T02:59:36.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.878+0000 7ff61e6a5700 1 -- 172.21.15.17:0/1080052102 wait complete. 2024-03-24T02:59:36.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff61e6a5700 1 Processor -- start 2024-03-24T02:59:36.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff61e6a5700 1 -- start start 2024-03-24T02:59:36.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff61e6a5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618101350 0x7ff618199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:36.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff61e6a5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff618101d00 0x7ff618199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:36.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff61e6a5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6181063a0 0x7ff6181a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:36.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff61e6a5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618111380 con 0x7ff618101d00 2024-03-24T02:59:36.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff61e6a5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618111200 con 0x7ff6181063a0 2024-03-24T02:59:36.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff61e6a5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618111500 con 0x7ff618101350 2024-03-24T02:59:36.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff6177fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff618101d00 0x7ff618199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:36.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff6177fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff618101d00 0x7ff618199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32936/0 (socket says 172.21.15.17:32936) 2024-03-24T02:59:36.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff6177fe700 1 -- 172.21.15.17:0/3970356401 learned_addr learned my addr 172.21.15.17:0/3970356401 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff617fff700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618101350 0x7ff618199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff61cc42700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6181063a0 0x7ff6181a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff6177fe700 1 -- 172.21.15.17:0/3970356401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618101350 msgr2=0x7ff618199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff6177fe700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618101350 0x7ff618199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff6177fe700 1 -- 172.21.15.17:0/3970356401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6181063a0 msgr2=0x7ff6181a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.879+0000 7ff6177fe700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6181063a0 0x7ff6181a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.880+0000 7ff6177fe700 1 -- 172.21.15.17:0/3970356401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6181a4d00 con 0x7ff618101d00 2024-03-24T02:59:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.880+0000 7ff61cc42700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6181063a0 0x7ff6181a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.880+0000 7ff6177fe700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff618101d00 0x7ff618199c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00d950 tx=0x7ff60c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.880+0000 7ff6157fa700 1 -- 172.21.15.17:0/3970356401 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c016070 con 0x7ff618101d00 2024-03-24T02:59:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.880+0000 7ff6157fa700 1 -- 172.21.15.17:0/3970356401 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff60c004d10 con 0x7ff618101d00 2024-03-24T02:59:36.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.880+0000 7ff61e6a5700 1 -- 172.21.15.17:0/3970356401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6181a4f30 con 0x7ff618101d00 2024-03-24T02:59:36.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.880+0000 7ff6157fa700 1 -- 172.21.15.17:0/3970356401 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c005600 con 0x7ff618101d00 2024-03-24T02:59:36.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.880+0000 7ff61e6a5700 1 -- 172.21.15.17:0/3970356401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6181a53e0 con 0x7ff618101d00 2024-03-24T02:59:36.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.881+0000 7ff61e6a5700 1 -- 172.21.15.17:0/3970356401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff618062860 con 0x7ff618101d00 2024-03-24T02:59:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.882+0000 7ff6157fa700 1 -- 172.21.15.17:0/3970356401 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff60c004220 con 0x7ff618101d00 2024-03-24T02:59:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.883+0000 7ff6157fa700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff6000a2920 0x7ff6000a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.883+0000 7ff617fff700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff6000a2920 0x7ff6000a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.883+0000 7ff6157fa700 1 -- 172.21.15.17:0/3970356401 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7ff60c0cf280 con 0x7ff618101d00 2024-03-24T02:59:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.883+0000 7ff6157fa700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517] conn(0x7ff6000ad980 0x7ff6000afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.883+0000 7ff6157fa700 1 -- 172.21.15.17:0/3970356401 --> [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff60c015ec0 con 0x7ff6000ad980 2024-03-24T02:59:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.883+0000 7ff6157fa700 1 -- 172.21.15.17:0/3970356401 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff60c0cf6d0 con 0x7ff618101d00 2024-03-24T02:59:36.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.883+0000 7ff61cc42700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517] conn(0x7ff6000ad980 0x7ff6000afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:36.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.883+0000 7ff617fff700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff6000a2920 0x7ff6000a4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff608000c00 tx=0x7ff608005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:36.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.884+0000 7ff61cc42700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517] conn(0x7ff6000ad980 0x7ff6000afe20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:36.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.885+0000 7ff6157fa700 1 -- 172.21.15.17:0/3970356401 <== osd.1 v2:172.21.15.19:6800/1476118517 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff60c015ec0 con 0x7ff6000ad980 2024-03-24T02:59:36.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.901+0000 7ff61e6a5700 1 -- 172.21.15.17:0/3970356401 --> [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff618062c90 con 0x7ff6000ad980 2024-03-24T02:59:36.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.902+0000 7ff6157fa700 1 -- 172.21.15.17:0/3970356401 <== osd.1 v2:172.21.15.19:6800/1476118517 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff618062c90 con 0x7ff6000ad980 2024-03-24T02:59:36.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.902+0000 7ff61e6a5700 1 -- 172.21.15.17:0/3970356401 >> [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517] conn(0x7ff6000ad980 msgr2=0x7ff6000afe20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:36.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.902+0000 7ff61e6a5700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517] conn(0x7ff6000ad980 0x7ff6000afe20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:36.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.902+0000 7ff61e6a5700 1 -- 172.21.15.17:0/3970356401 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff6000a2920 msgr2=0x7ff6000a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:36.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.902+0000 7ff61e6a5700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff6000a2920 0x7ff6000a4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff608000c00 tx=0x7ff608005fb0 comp rx=0 tx=0).stop 2024-03-24T02:59:36.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.902+0000 7ff61e6a5700 1 -- 172.21.15.17:0/3970356401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff618101d00 msgr2=0x7ff618199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:36.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.902+0000 7ff61e6a5700 1 --2- 172.21.15.17:0/3970356401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff618101d00 0x7ff618199c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00d950 tx=0x7ff60c00dc60 comp rx=0 tx=0).stop 2024-03-24T02:59:36.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.902+0000 7ff61cc42700 1 -- 172.21.15.17:0/3970356401 reap_dead start 2024-03-24T02:59:36.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.902+0000 7ff61e6a5700 1 -- 172.21.15.17:0/3970356401 shutdown_connections 2024-03-24T02:59:36.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.902+0000 7ff61e6a5700 1 -- 172.21.15.17:0/3970356401 >> 172.21.15.17:0/3970356401 conn(0x7ff6180770e0 msgr2=0x7ff618100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:36.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.902+0000 7ff61e6a5700 1 -- 172.21.15.17:0/3970356401 shutdown_connections 2024-03-24T02:59:36.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:36.902+0000 7ff61e6a5700 1 -- 172.21.15.17:0/3970356401 wait complete. 2024-03-24T02:59:37.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.090+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/468332350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e841081e0 msgr2=0x7f6e841085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:37.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.090+0000 7f6e8a6dc700 1 --2- 172.21.15.17:0/468332350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e841081e0 0x7f6e841085c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6e740099c0 tx=0x7f6e74009cd0 comp rx=0 tx=0).stop 2024-03-24T02:59:37.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.090+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/468332350 shutdown_connections 2024-03-24T02:59:37.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.090+0000 7f6e8a6dc700 1 --2- 172.21.15.17:0/468332350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e8406a7e0 0x7f6e8410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:37.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.090+0000 7f6e8a6dc700 1 --2- 172.21.15.17:0/468332350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e841081e0 0x7f6e841085c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:37.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.090+0000 7f6e8a6dc700 1 --2- 172.21.15.17:0/468332350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e84069e40 0x7f6e8406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:37.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.090+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/468332350 >> 172.21.15.17:0/468332350 conn(0x7f6e84077fc0 msgr2=0x7f6e840783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:37.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.090+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/468332350 shutdown_connections 2024-03-24T02:59:37.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.090+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/468332350 wait complete. 2024-03-24T02:59:37.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.091+0000 7f6e8a6dc700 1 Processor -- start 2024-03-24T02:59:37.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.091+0000 7f6e8a6dc700 1 -- start start 2024-03-24T02:59:37.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.091+0000 7f6e8a6dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e84069e40 0x7f6e84101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:37.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.091+0000 7f6e8a6dc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e8406a7e0 0x7f6e841016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:37.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.091+0000 7f6e8a6dc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e841081e0 0x7f6e84101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:37.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.091+0000 7f6e8a6dc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e84112420 con 0x7f6e8406a7e0 2024-03-24T02:59:37.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.091+0000 7f6e8a6dc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e841122a0 con 0x7f6e84069e40 2024-03-24T02:59:37.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.091+0000 7f6e8a6dc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e841125a0 con 0x7f6e841081e0 2024-03-24T02:59:37.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e837fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e8406a7e0 0x7f6e841016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:37.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e83fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e84069e40 0x7f6e84101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:37.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e837fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e8406a7e0 0x7f6e841016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32958/0 (socket says 172.21.15.17:32958) 2024-03-24T02:59:37.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e837fe700 1 -- 172.21.15.17:0/3514170046 learned_addr learned my addr 172.21.15.17:0/3514170046 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:37.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e837fe700 1 -- 172.21.15.17:0/3514170046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e841081e0 msgr2=0x7f6e84101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:37.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e837fe700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e841081e0 0x7f6e84101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:37.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e837fe700 1 -- 172.21.15.17:0/3514170046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e84069e40 msgr2=0x7f6e84101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:37.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e837fe700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e84069e40 0x7f6e84101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:37.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e837fe700 1 -- 172.21.15.17:0/3514170046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e841a6de0 con 0x7f6e8406a7e0 2024-03-24T02:59:37.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e83fff700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e84069e40 0x7f6e84101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:37.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e837fe700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e8406a7e0 0x7f6e841016a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800d950 tx=0x7f6e7800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:37.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e817fa700 1 -- 172.21.15.17:0/3514170046 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78016070 con 0x7f6e8406a7e0 2024-03-24T02:59:37.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e817fa700 1 -- 172.21.15.17:0/3514170046 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e78009e20 con 0x7f6e8406a7e0 2024-03-24T02:59:37.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/3514170046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e841a70d0 con 0x7f6e8406a7e0 2024-03-24T02:59:37.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.092+0000 7f6e817fa700 1 -- 172.21.15.17:0/3514170046 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7800f3c0 con 0x7f6e8406a7e0 2024-03-24T02:59:37.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.093+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/3514170046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e841a75c0 con 0x7f6e8406a7e0 2024-03-24T02:59:37.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.093+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/3514170046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e840628a0 con 0x7f6e8406a7e0 2024-03-24T02:59:37.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.095+0000 7f6e817fa700 1 -- 172.21.15.17:0/3514170046 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f6e78019070 con 0x7f6e8406a7e0 2024-03-24T02:59:37.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.095+0000 7f6e817fa700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6e6c0a2920 0x7f6e6c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:37.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.095+0000 7f6e817fa700 1 -- 172.21.15.17:0/3514170046 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f6e780cef80 con 0x7f6e8406a7e0 2024-03-24T02:59:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.095+0000 7f6e83fff700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6e6c0a2920 0x7f6e6c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.095+0000 7f6e817fa700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354] conn(0x7f6e6c0ad980 0x7f6e6c0afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.095+0000 7f6e817fa700 1 -- 172.21.15.17:0/3514170046 --> [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e78014e80 con 0x7f6e6c0ad980 2024-03-24T02:59:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.095+0000 7f6e817fa700 1 -- 172.21.15.17:0/3514170046 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e780cf3d0 con 0x7f6e8406a7e0 2024-03-24T02:59:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.096+0000 7f6e88c79700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354] conn(0x7f6e6c0ad980 0x7f6e6c0afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.096+0000 7f6e83fff700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6e6c0a2920 0x7f6e6c0a4de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6e74006010 tx=0x7f6e74005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.096+0000 7f6e88c79700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354] conn(0x7f6e6c0ad980 0x7f6e6c0afe20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.097+0000 7f6e817fa700 1 -- 172.21.15.17:0/3514170046 <== osd.6 v2:172.21.15.17:6818/3664721354 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6e78014e80 con 0x7f6e6c0ad980 2024-03-24T02:59:37.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.115+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/3514170046 --> [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6e84062d20 con 0x7f6e6c0ad980 2024-03-24T02:59:37.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.115+0000 7f6e817fa700 1 -- 172.21.15.17:0/3514170046 <== osd.6 v2:172.21.15.17:6818/3664721354 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6e84062d20 con 0x7f6e6c0ad980 2024-03-24T02:59:37.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.115+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354] conn(0x7f6e6c0ad980 msgr2=0x7f6e6c0afe20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:37.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.115+0000 7f6e8a6dc700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354] conn(0x7f6e6c0ad980 0x7f6e6c0afe20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:37.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.116+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6e6c0a2920 msgr2=0x7f6e6c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:37.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.116+0000 7f6e8a6dc700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f6e6c0a2920 0x7f6e6c0a4de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6e74006010 tx=0x7f6e74005960 comp rx=0 tx=0).stop 2024-03-24T02:59:37.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.116+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e8406a7e0 msgr2=0x7f6e841016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:37.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.116+0000 7f6e8a6dc700 1 --2- 172.21.15.17:0/3514170046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e8406a7e0 0x7f6e841016a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800d950 tx=0x7f6e7800dc60 comp rx=0 tx=0).stop 2024-03-24T02:59:37.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.116+0000 7f6e88c79700 1 -- 172.21.15.17:0/3514170046 reap_dead start 2024-03-24T02:59:37.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.116+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/3514170046 shutdown_connections 2024-03-24T02:59:37.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.116+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/3514170046 >> 172.21.15.17:0/3514170046 conn(0x7f6e84077fc0 msgr2=0x7f6e840ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:37.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.116+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/3514170046 shutdown_connections 2024-03-24T02:59:37.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:37.116+0000 7f6e8a6dc700 1 -- 172.21.15.17:0/3514170046 wait complete. 2024-03-24T02:59:37.772 INFO:teuthology.orchestra.run.smithi017.stdout:55834574925 2024-03-24T02:59:37.772 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-24T02:59:38.435 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:59:38.435 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:59:38.435 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:59:38.435 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:38 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:59:38.435 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:38 smithi017 ceph-mon[31926]: pgmap v325: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 9 op/s 2024-03-24T02:59:38.460 INFO:teuthology.orchestra.run.smithi017.stdout:287762808867 2024-03-24T02:59:38.461 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-24T02:59:38.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:59:38.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:59:38.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:59:38.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:38 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:59:38.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:38 smithi188 ceph-mon[36206]: pgmap v325: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 9 op/s 2024-03-24T02:59:38.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:59:38.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:59:38.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:59:38.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:38 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T02:59:38.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:38 smithi019 ceph-mon[35944]: pgmap v325: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 9 op/s 2024-03-24T02:59:38.863 INFO:teuthology.orchestra.run.smithi017.stdout:309237645343 2024-03-24T02:59:38.863 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-24T02:59:39.504 INFO:teuthology.orchestra.run.smithi017.stdout:34359738448 2024-03-24T02:59:39.505 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-24T02:59:40.024 INFO:teuthology.orchestra.run.smithi017.stdout:197568495668 2024-03-24T02:59:40.025 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-24T02:59:40.127 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:40 smithi017 ceph-mon[31926]: pgmap v326: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-24T02:59:40.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:40 smithi188 ceph-mon[36206]: pgmap v326: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-24T02:59:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:40 smithi019 ceph-mon[35944]: pgmap v326: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-24T02:59:40.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.530+0000 7f779ba09700 1 -- 172.21.15.17:0/2998055336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7794102ba0 msgr2=0x7f779410efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:40.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.530+0000 7f779ba09700 1 --2- 172.21.15.17:0/2998055336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7794102ba0 0x7f779410efe0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f778c00b0d0 tx=0x7f778c00b3e0 comp rx=0 tx=0).stop 2024-03-24T02:59:40.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.530+0000 7f779ba09700 1 -- 172.21.15.17:0/2998055336 shutdown_connections 2024-03-24T02:59:40.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.530+0000 7f779ba09700 1 --2- 172.21.15.17:0/2998055336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7794102ba0 0x7f779410efe0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:40.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.530+0000 7f779ba09700 1 --2- 172.21.15.17:0/2998055336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77941081c0 0x7f77941085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:40.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.530+0000 7f779ba09700 1 --2- 172.21.15.17:0/2998055336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77941021e0 0x7f7794102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:40.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.530+0000 7f779ba09700 1 -- 172.21.15.17:0/2998055336 >> 172.21.15.17:0/2998055336 conn(0x7f77940792c0 msgr2=0x7f77940796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:40.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.531+0000 7f779ba09700 1 -- 172.21.15.17:0/2998055336 shutdown_connections 2024-03-24T02:59:40.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.531+0000 7f779ba09700 1 -- 172.21.15.17:0/2998055336 wait complete. 2024-03-24T02:59:40.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.531+0000 7f779ba09700 1 Processor -- start 2024-03-24T02:59:40.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.531+0000 7f779ba09700 1 -- start start 2024-03-24T02:59:40.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.531+0000 7f779ba09700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77941021e0 0x7f7794074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:40.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.531+0000 7f779ba09700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7794102ba0 0x7f77940749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:40.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.531+0000 7f779ba09700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77941081c0 0x7f7794074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:40.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.531+0000 7f779ba09700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7794112470 con 0x7f77941081c0 2024-03-24T02:59:40.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.531+0000 7f779ba09700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77941122f0 con 0x7f7794102ba0 2024-03-24T02:59:40.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.531+0000 7f779ba09700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77941125f0 con 0x7f77941021e0 2024-03-24T02:59:40.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f7799fa6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77941081c0 0x7f7794074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:40.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f7798fa4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7794102ba0 0x7f77940749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:40.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f7799fa6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77941081c0 0x7f7794074f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32970/0 (socket says 172.21.15.17:32970) 2024-03-24T02:59:40.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f77997a5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77941021e0 0x7f7794074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:40.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f7799fa6700 1 -- 172.21.15.17:0/1680918715 learned_addr learned my addr 172.21.15.17:0/1680918715 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:40.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f7798fa4700 1 -- 172.21.15.17:0/1680918715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77941021e0 msgr2=0x7f7794074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:40.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f7798fa4700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77941021e0 0x7f7794074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:40.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f7798fa4700 1 -- 172.21.15.17:0/1680918715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77941081c0 msgr2=0x7f7794074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:40.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f7798fa4700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77941081c0 0x7f7794074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:40.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f7798fa4700 1 -- 172.21.15.17:0/1680918715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77941a6e80 con 0x7f7794102ba0 2024-03-24T02:59:40.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f7799fa6700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77941081c0 0x7f7794074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:40.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f7798fa4700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7794102ba0 0x7f77940749d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9f0 tx=0x7f778400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:40.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f778a7fc700 1 -- 172.21.15.17:0/1680918715 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784004220 con 0x7f7794102ba0 2024-03-24T02:59:40.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f779ba09700 1 -- 172.21.15.17:0/1680918715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77941a7170 con 0x7f7794102ba0 2024-03-24T02:59:40.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f778a7fc700 1 -- 172.21.15.17:0/1680918715 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7784004d10 con 0x7f7794102ba0 2024-03-24T02:59:40.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f778a7fc700 1 -- 172.21.15.17:0/1680918715 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784005020 con 0x7f7794102ba0 2024-03-24T02:59:40.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.532+0000 7f779ba09700 1 -- 172.21.15.17:0/1680918715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77941a7660 con 0x7f7794102ba0 2024-03-24T02:59:40.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.533+0000 7f779ba09700 1 -- 172.21.15.17:0/1680918715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7794062860 con 0x7f7794102ba0 2024-03-24T02:59:40.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.534+0000 7f778a7fc700 1 -- 172.21.15.17:0/1680918715 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f7784004750 con 0x7f7794102ba0 2024-03-24T02:59:40.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.535+0000 7f778a7fc700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f77800a2920 0x7f77800a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:40.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.535+0000 7f77997a5700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f77800a2920 0x7f77800a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:40.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.535+0000 7f778a7fc700 1 -- 172.21.15.17:0/1680918715 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f77840cf0d0 con 0x7f7794102ba0 2024-03-24T02:59:40.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.535+0000 7f778a7fc700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507] conn(0x7f77800ad980 0x7f77800afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:40.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.535+0000 7f778a7fc700 1 -- 172.21.15.17:0/1680918715 --> [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7784012ec0 con 0x7f77800ad980 2024-03-24T02:59:40.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.535+0000 7f778a7fc700 1 -- 172.21.15.17:0/1680918715 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77840cf480 con 0x7f7794102ba0 2024-03-24T02:59:40.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.535+0000 7f7799fa6700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507] conn(0x7f77800ad980 0x7f77800afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:40.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.535+0000 7f77997a5700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f77800a2920 0x7f77800a4de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7790009be0 tx=0x7f7790009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:40.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.536+0000 7f7799fa6700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507] conn(0x7f77800ad980 0x7f77800afe20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:40.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.537+0000 7f778a7fc700 1 -- 172.21.15.17:0/1680918715 <== osd.7 v2:172.21.15.19:6816/2743063507 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7784012ec0 con 0x7f77800ad980 2024-03-24T02:59:40.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.555+0000 7f779ba09700 1 -- 172.21.15.17:0/1680918715 --> [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7794062c60 con 0x7f77800ad980 2024-03-24T02:59:40.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.556+0000 7f778a7fc700 1 -- 172.21.15.17:0/1680918715 <== osd.7 v2:172.21.15.19:6816/2743063507 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7794062c60 con 0x7f77800ad980 2024-03-24T02:59:40.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.556+0000 7f779ba09700 1 -- 172.21.15.17:0/1680918715 >> [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507] conn(0x7f77800ad980 msgr2=0x7f77800afe20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:40.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.556+0000 7f779ba09700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507] conn(0x7f77800ad980 0x7f77800afe20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:40.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.556+0000 7f779ba09700 1 -- 172.21.15.17:0/1680918715 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f77800a2920 msgr2=0x7f77800a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:40.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.556+0000 7f779ba09700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f77800a2920 0x7f77800a4de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7790009be0 tx=0x7f7790009260 comp rx=0 tx=0).stop 2024-03-24T02:59:40.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.556+0000 7f779ba09700 1 -- 172.21.15.17:0/1680918715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7794102ba0 msgr2=0x7f77940749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:40.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.556+0000 7f779ba09700 1 --2- 172.21.15.17:0/1680918715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7794102ba0 0x7f77940749d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9f0 tx=0x7f778400ed00 comp rx=0 tx=0).stop 2024-03-24T02:59:40.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.556+0000 7f7799fa6700 1 -- 172.21.15.17:0/1680918715 reap_dead start 2024-03-24T02:59:40.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.557+0000 7f779ba09700 1 -- 172.21.15.17:0/1680918715 shutdown_connections 2024-03-24T02:59:40.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.557+0000 7f779ba09700 1 -- 172.21.15.17:0/1680918715 >> 172.21.15.17:0/1680918715 conn(0x7f77940792c0 msgr2=0x7f7794100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:40.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.557+0000 7f779ba09700 1 -- 172.21.15.17:0/1680918715 shutdown_connections 2024-03-24T02:59:40.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:40.557+0000 7f779ba09700 1 -- 172.21.15.17:0/1680918715 wait complete. 2024-03-24T02:59:40.580 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:40.832 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:41.570 INFO:teuthology.orchestra.run.smithi017.stdout:219043332143 2024-03-24T02:59:41.571 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-24T02:59:41.994 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:42.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:42 smithi188 ceph-mon[36206]: pgmap v327: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-24T02:59:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:42 smithi019 ceph-mon[35944]: pgmap v327: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-24T02:59:42.450 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:42 smithi017 ceph-mon[31926]: pgmap v327: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-24T02:59:43.983 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:43.988 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:43.988 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:43.988 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:43.990 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:43.993 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:43.994 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:44.013 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:45.129 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:45 smithi017 ceph-mon[31926]: pgmap v328: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-24T02:59:45.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:45 smithi188 ceph-mon[36206]: pgmap v328: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-24T02:59:45.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:45 smithi019 ceph-mon[35944]: pgmap v328: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-24T02:59:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.979+0000 7f37a21fd700 1 -- 172.21.15.17:0/2160897109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f379c101cd0 msgr2=0x7f379c105cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.979+0000 7f37a21fd700 1 --2- 172.21.15.17:0/2160897109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f379c101cd0 0x7f379c105cd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3790009990 tx=0x7f3790009ca0 comp rx=0 tx=0).stop 2024-03-24T02:59:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.979+0000 7f37a21fd700 1 -- 172.21.15.17:0/2160897109 shutdown_connections 2024-03-24T02:59:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.979+0000 7f37a21fd700 1 --2- 172.21.15.17:0/2160897109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c1063b0 0x7f379c10df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.979+0000 7f37a21fd700 1 --2- 172.21.15.17:0/2160897109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f379c101320 0x7f379c101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.979+0000 7f37a21fd700 1 --2- 172.21.15.17:0/2160897109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f379c101cd0 0x7f379c105cd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:45.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.979+0000 7f37a21fd700 1 -- 172.21.15.17:0/2160897109 >> 172.21.15.17:0/2160897109 conn(0x7f379c0770b0 msgr2=0x7f379c0774c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.979+0000 7f37a21fd700 1 -- 172.21.15.17:0/2160897109 shutdown_connections 2024-03-24T02:59:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.979+0000 7f37a21fd700 1 -- 172.21.15.17:0/2160897109 wait complete. 2024-03-24T02:59:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.979+0000 7f37a21fd700 1 Processor -- start 2024-03-24T02:59:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.979+0000 7f37a21fd700 1 -- start start 2024-03-24T02:59:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f37a21fd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f379c101320 0x7f379c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f37a21fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c101cd0 0x7f379c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f37a21fd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f379c1063b0 0x7f379c1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f37a21fd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c1113a0 con 0x7f379c101320 2024-03-24T02:59:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f37a21fd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c111220 con 0x7f379c101cd0 2024-03-24T02:59:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f37a21fd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c111520 con 0x7f379c1063b0 2024-03-24T02:59:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f379b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f379c101320 0x7f379c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:45.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f379affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c101cd0 0x7f379c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f379b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f379c101320 0x7f379c19d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33320/0 (socket says 172.21.15.17:33320) 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f379b7fe700 1 -- 172.21.15.17:0/20282706 learned_addr learned my addr 172.21.15.17:0/20282706 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f379bfff700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f379c1063b0 0x7f379c1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f379b7fe700 1 -- 172.21.15.17:0/20282706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f379c1063b0 msgr2=0x7f379c1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f379b7fe700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f379c1063b0 0x7f379c1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f379b7fe700 1 -- 172.21.15.17:0/20282706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c101cd0 msgr2=0x7f379c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f379b7fe700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c101cd0 0x7f379c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f379b7fe700 1 -- 172.21.15.17:0/20282706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c197fc0 con 0x7f379c101320 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f379affd700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c101cd0 0x7f379c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.980+0000 7f379b7fe700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f379c101320 0x7f379c19d610 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f378c00e9a0 tx=0x7f378c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.981+0000 7f3798ff9700 1 -- 172.21.15.17:0/20282706 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c00ca10 con 0x7f379c101320 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.981+0000 7f3798ff9700 1 -- 172.21.15.17:0/20282706 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f378c00cbb0 con 0x7f379c101320 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.981+0000 7f37a21fd700 1 -- 172.21.15.17:0/20282706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c1982b0 con 0x7f379c101320 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.981+0000 7f3798ff9700 1 -- 172.21.15.17:0/20282706 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c018680 con 0x7f379c101320 2024-03-24T02:59:45.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.981+0000 7f37a21fd700 1 -- 172.21.15.17:0/20282706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c1a4590 con 0x7f379c101320 2024-03-24T02:59:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.982+0000 7f37a21fd700 1 -- 172.21.15.17:0/20282706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c1027b0 con 0x7f379c101320 2024-03-24T02:59:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.983+0000 7f3798ff9700 1 -- 172.21.15.17:0/20282706 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f378c010430 con 0x7f379c101320 2024-03-24T02:59:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.983+0000 7f3798ff9700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f37840a29a0 0x7f37840a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.983+0000 7f379affd700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f37840a29a0 0x7f37840a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.983+0000 7f3798ff9700 1 -- 172.21.15.17:0/20282706 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f378c0cf130 con 0x7f379c101320 2024-03-24T02:59:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.984+0000 7f379affd700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f37840a29a0 0x7f37840a4e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f379c198f30 tx=0x7f3790005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:45.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:45.989+0000 7f3798ff9700 1 -- 172.21.15.17:0/20282706 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f378c09bce0 con 0x7f379c101320 2024-03-24T02:59:46.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.202+0000 7f37a21fd700 1 -- 172.21.15.17:0/20282706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f379c198c10 con 0x7f379c101320 2024-03-24T02:59:46.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.203+0000 7f3798ff9700 1 -- 172.21.15.17:0/20282706 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f378c09b5e0 con 0x7f379c101320 2024-03-24T02:59:46.203 INFO:teuthology.orchestra.run.smithi017.stdout:141733920832 2024-03-24T02:59:46.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.205+0000 7f37a21fd700 1 -- 172.21.15.17:0/20282706 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f37840a29a0 msgr2=0x7f37840a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:46.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.205+0000 7f37a21fd700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f37840a29a0 0x7f37840a4e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f379c198f30 tx=0x7f3790005fd0 comp rx=0 tx=0).stop 2024-03-24T02:59:46.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.205+0000 7f37a21fd700 1 -- 172.21.15.17:0/20282706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f379c101320 msgr2=0x7f379c19d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:46.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.205+0000 7f37a21fd700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f379c101320 0x7f379c19d610 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f378c00e9a0 tx=0x7f378c00ecb0 comp rx=0 tx=0).stop 2024-03-24T02:59:46.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.205+0000 7f37a21fd700 1 -- 172.21.15.17:0/20282706 shutdown_connections 2024-03-24T02:59:46.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.205+0000 7f37a21fd700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f37840a29a0 0x7f37840a4e60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:46.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.205+0000 7f37a21fd700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f379c1063b0 0x7f379c1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:46.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.205+0000 7f37a21fd700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f379c101320 0x7f379c19d610 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:46.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.205+0000 7f37a21fd700 1 --2- 172.21.15.17:0/20282706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c101cd0 0x7f379c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:46.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.205+0000 7f37a21fd700 1 -- 172.21.15.17:0/20282706 >> 172.21.15.17:0/20282706 conn(0x7f379c0770b0 msgr2=0x7f379c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:46.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.205+0000 7f37a21fd700 1 -- 172.21.15.17:0/20282706 shutdown_connections 2024-03-24T02:59:46.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:46.205+0000 7f37a21fd700 1 -- 172.21.15.17:0/20282706 wait complete. 2024-03-24T02:59:47.089 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:47 smithi017 ceph-mon[31926]: pgmap v329: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T02:59:47.090 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:47 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/20282706' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T02:59:47.091 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920828 got 141733920832 for osd.4 2024-03-24T02:59:47.091 DEBUG:teuthology.parallel:result is None 2024-03-24T02:59:47.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:47 smithi188 ceph-mon[36206]: pgmap v329: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T02:59:47.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:47 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/20282706' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T02:59:47.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:47 smithi019 ceph-mon[35944]: pgmap v329: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T02:59:47.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:47 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/20282706' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T02:59:49.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:49 smithi188 ceph-mon[36206]: pgmap v330: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-24T02:59:49.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:49 smithi019 ceph-mon[35944]: pgmap v330: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-24T02:59:49.406 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:49 smithi017 ceph-mon[31926]: pgmap v330: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-24T02:59:50.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.129+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/4255020793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f18069e40 msgr2=0x7f9f1806a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:50.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.129+0000 7f9f1e8a6700 1 --2- 172.21.15.17:0/4255020793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f18069e40 0x7f9f1806a2a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0098f0 tx=0x7f9f0c009c00 comp rx=0 tx=0).stop 2024-03-24T02:59:50.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.129+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/4255020793 shutdown_connections 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.129+0000 7f9f1e8a6700 1 --2- 172.21.15.17:0/4255020793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f1806a7e0 0x7f9f1810f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.129+0000 7f9f1e8a6700 1 --2- 172.21.15.17:0/4255020793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f181081e0 0x7f9f181085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.129+0000 7f9f1e8a6700 1 --2- 172.21.15.17:0/4255020793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f18069e40 0x7f9f1806a2a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.129+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/4255020793 >> 172.21.15.17:0/4255020793 conn(0x7f9f18077fc0 msgr2=0x7f9f180783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.130+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/4255020793 shutdown_connections 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.130+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/4255020793 wait complete. 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.130+0000 7f9f1e8a6700 1 Processor -- start 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.130+0000 7f9f1e8a6700 1 -- start start 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f1e8a6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f18069e40 0x7f9f1819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f1e8a6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1806a7e0 0x7f9f1819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f1e8a6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f181081e0 0x7f9f181987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f1e8a6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f18112490 con 0x7f9f1806a7e0 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f1e8a6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f18112310 con 0x7f9f181081e0 2024-03-24T02:59:50.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f1e8a6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f18112610 con 0x7f9f18069e40 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f177fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1806a7e0 0x7f9f1819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f177fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1806a7e0 0x7f9f1819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33332/0 (socket says 172.21.15.17:33332) 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f177fe700 1 -- 172.21.15.17:0/619714151 learned_addr learned my addr 172.21.15.17:0/619714151 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f1ce43700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f181081e0 0x7f9f181987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f17fff700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f18069e40 0x7f9f1819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f1ce43700 1 -- 172.21.15.17:0/619714151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f18069e40 msgr2=0x7f9f1819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f1ce43700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f18069e40 0x7f9f1819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f1ce43700 1 -- 172.21.15.17:0/619714151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1806a7e0 msgr2=0x7f9f1819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f1ce43700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1806a7e0 0x7f9f1819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f1ce43700 1 -- 172.21.15.17:0/619714151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f18198f90 con 0x7f9f181081e0 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.131+0000 7f9f177fe700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1806a7e0 0x7f9f1819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.132+0000 7f9f1ce43700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f181081e0 0x7f9f181987b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000efb0 tx=0x7f9f1000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:50.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.132+0000 7f9f157fa700 1 -- 172.21.15.17:0/619714151 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10003a00 con 0x7f9f181081e0 2024-03-24T02:59:50.602 INFO:teuthology.orchestra.run.smithi017.stdout:343597383709 2024-03-24T02:59:50.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.132+0000 7f9f157fa700 1 -- 172.21.15.17:0/619714151 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f10003ba0 con 0x7f9f181081e0 2024-03-24T02:59:50.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.132+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/619714151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f18199280 con 0x7f9f181081e0 2024-03-24T02:59:50.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.132+0000 7f9f157fa700 1 -- 172.21.15.17:0/619714151 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10004850 con 0x7f9f181081e0 2024-03-24T02:59:50.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.132+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/619714151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f181a5490 con 0x7f9f181081e0 2024-03-24T02:59:50.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.133+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/619714151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f18103890 con 0x7f9f181081e0 2024-03-24T02:59:50.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.134+0000 7f9f157fa700 1 -- 172.21.15.17:0/619714151 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f9f10012050 con 0x7f9f181081e0 2024-03-24T02:59:50.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.134+0000 7f9f157fa700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9f000a2920 0x7f9f000a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.135+0000 7f9f17fff700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9f000a2920 0x7f9f000a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.135+0000 7f9f157fa700 1 -- 172.21.15.17:0/619714151 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f9f100ce830 con 0x7f9f181081e0 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.135+0000 7f9f17fff700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9f000a2920 0x7f9f000a4de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009750 tx=0x7f9f08009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.144+0000 7f9f157fa700 1 -- 172.21.15.17:0/619714151 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9f10022060 con 0x7f9f181081e0 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.352+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/619714151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f9f1802dbf0 con 0x7f9f181081e0 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.353+0000 7f9f157fa700 1 -- 172.21.15.17:0/619714151 <== mon.2 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f9f10010040 con 0x7f9f181081e0 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.355+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/619714151 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9f000a2920 msgr2=0x7f9f000a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.355+0000 7f9f1e8a6700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9f000a2920 0x7f9f000a4de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009750 tx=0x7f9f08009310 comp rx=0 tx=0).stop 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.355+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/619714151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f181081e0 msgr2=0x7f9f181987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.355+0000 7f9f1e8a6700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f181081e0 0x7f9f181987b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000efb0 tx=0x7f9f1000c3a0 comp rx=0 tx=0).stop 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.355+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/619714151 shutdown_connections 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.355+0000 7f9f1e8a6700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f9f000a2920 0x7f9f000a4de0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.355+0000 7f9f1e8a6700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f181081e0 0x7f9f181987b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.355+0000 7f9f1e8a6700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f18069e40 0x7f9f1819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:50.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.355+0000 7f9f1e8a6700 1 --2- 172.21.15.17:0/619714151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1806a7e0 0x7f9f1819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:50.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.355+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/619714151 >> 172.21.15.17:0/619714151 conn(0x7f9f18077fc0 msgr2=0x7f9f180ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:50.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.355+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/619714151 shutdown_connections 2024-03-24T02:59:50.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:50.355+0000 7f9f1e8a6700 1 -- 172.21.15.17:0/619714151 wait complete. 2024-03-24T02:59:51.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:51 smithi017 ceph-mon[31926]: pgmap v331: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-03-24T02:59:51.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:51 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/619714151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-24T02:59:51.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:51 smithi188 ceph-mon[36206]: pgmap v331: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-03-24T02:59:51.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:51 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/619714151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-24T02:59:51.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:51 smithi019 ceph-mon[35944]: pgmap v331: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-03-24T02:59:51.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:51 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/619714151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-24T02:59:51.912 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383703 got 343597383709 for osd.11 2024-03-24T02:59:51.913 DEBUG:teuthology.parallel:result is None 2024-03-24T02:59:52.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.075+0000 7f66c5e4a700 1 -- 172.21.15.17:0/353621093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c0107280 msgr2=0x7f66c0107660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:52.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.075+0000 7f66c5e4a700 1 --2- 172.21.15.17:0/353621093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c0107280 0x7f66c0107660 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009990 tx=0x7f66b0009ca0 comp rx=0 tx=0).stop 2024-03-24T02:59:52.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.075+0000 7f66c5e4a700 1 -- 172.21.15.17:0/353621093 shutdown_connections 2024-03-24T02:59:52.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.075+0000 7f66c5e4a700 1 --2- 172.21.15.17:0/353621093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66c0101d30 0x7f66c010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:52.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.075+0000 7f66c5e4a700 1 --2- 172.21.15.17:0/353621093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c0107280 0x7f66c0107660 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:52.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.075+0000 7f66c5e4a700 1 --2- 172.21.15.17:0/353621093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c0101370 0x7f66c01017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:52.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.075+0000 7f66c5e4a700 1 -- 172.21.15.17:0/353621093 >> 172.21.15.17:0/353621093 conn(0x7f66c0077ef0 msgr2=0x7f66c0078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:52.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.075+0000 7f66c5e4a700 1 -- 172.21.15.17:0/353621093 shutdown_connections 2024-03-24T02:59:52.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.075+0000 7f66c5e4a700 1 -- 172.21.15.17:0/353621093 wait complete. 2024-03-24T02:59:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66c5e4a700 1 Processor -- start 2024-03-24T02:59:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66c5e4a700 1 -- start start 2024-03-24T02:59:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66c5e4a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c0101370 0x7f66c019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66c5e4a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66c0101d30 0x7f66c019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66c5e4a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c0107280 0x7f66c01965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66c5e4a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c01102f0 con 0x7f66c0101370 2024-03-24T02:59:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66c5e4a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c0110170 con 0x7f66c0107280 2024-03-24T02:59:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66c5e4a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c0110470 con 0x7f66c0101d30 2024-03-24T02:59:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66bf7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c0101370 0x7f66c019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66bf7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c0101370 0x7f66c019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54054/0 (socket says 172.21.15.17:54054) 2024-03-24T02:59:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66bf7fe700 1 -- 172.21.15.17:0/2385375255 learned_addr learned my addr 172.21.15.17:0/2385375255 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66bffff700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c0107280 0x7f66c01965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.076+0000 7f66beffd700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66c0101d30 0x7f66c019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.077+0000 7f66bf7fe700 1 -- 172.21.15.17:0/2385375255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66c0101d30 msgr2=0x7f66c019c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.077+0000 7f66bf7fe700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66c0101d30 0x7f66c019c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.077+0000 7f66bf7fe700 1 -- 172.21.15.17:0/2385375255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c0107280 msgr2=0x7f66c01965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:52.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.077+0000 7f66bf7fe700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c0107280 0x7f66c01965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:52.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.077+0000 7f66bf7fe700 1 -- 172.21.15.17:0/2385375255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c0196d30 con 0x7f66c0101370 2024-03-24T02:59:52.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.077+0000 7f66bf7fe700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c0101370 0x7f66c019c410 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f66b0000c00 tx=0x7f66b000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:52.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.077+0000 7f66bcff9700 1 -- 172.21.15.17:0/2385375255 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0027070 con 0x7f66c0101370 2024-03-24T02:59:52.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.077+0000 7f66bcff9700 1 -- 172.21.15.17:0/2385375255 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66b0003d70 con 0x7f66c0101370 2024-03-24T02:59:52.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.077+0000 7f66c5e4a700 1 -- 172.21.15.17:0/2385375255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c0196fc0 con 0x7f66c0101370 2024-03-24T02:59:52.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.077+0000 7f66bcff9700 1 -- 172.21.15.17:0/2385375255 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b000f460 con 0x7f66c0101370 2024-03-24T02:59:52.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.077+0000 7f66c5e4a700 1 -- 172.21.15.17:0/2385375255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c01a32e0 con 0x7f66c0101370 2024-03-24T02:59:52.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.078+0000 7f66c5e4a700 1 -- 172.21.15.17:0/2385375255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c0102810 con 0x7f66c0101370 2024-03-24T02:59:52.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.080+0000 7f66bcff9700 1 -- 172.21.15.17:0/2385375255 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f66b002a070 con 0x7f66c0101370 2024-03-24T02:59:52.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.080+0000 7f66bcff9700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f66a80a29a0 0x7f66a80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:52.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.080+0000 7f66beffd700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f66a80a29a0 0x7f66a80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:52.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.081+0000 7f66bcff9700 1 -- 172.21.15.17:0/2385375255 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f66b00d0050 con 0x7f66c0101370 2024-03-24T02:59:52.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.081+0000 7f66beffd700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f66a80a29a0 0x7f66a80a4e60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f66c0078410 tx=0x7f66b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:52.082 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T02:59:52.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.087+0000 7f66bcff9700 1 -- 172.21.15.17:0/2385375255 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f66b0013070 con 0x7f66c0101370 2024-03-24T02:59:52.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.299+0000 7f66c5e4a700 1 -- 172.21.15.17:0/2385375255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f66c0197b00 con 0x7f66c0101370 2024-03-24T02:59:52.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.299+0000 7f66bcff9700 1 -- 172.21.15.17:0/2385375255 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f66b009ccf0 con 0x7f66c0101370 2024-03-24T02:59:52.299 INFO:teuthology.orchestra.run.smithi017.stdout:287762808872 2024-03-24T02:59:52.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.301+0000 7f66c5e4a700 1 -- 172.21.15.17:0/2385375255 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f66a80a29a0 msgr2=0x7f66a80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:52.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.301+0000 7f66c5e4a700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f66a80a29a0 0x7f66a80a4e60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f66c0078410 tx=0x7f66b4009260 comp rx=0 tx=0).stop 2024-03-24T02:59:52.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.301+0000 7f66c5e4a700 1 -- 172.21.15.17:0/2385375255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c0101370 msgr2=0x7f66c019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:52.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.301+0000 7f66c5e4a700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c0101370 0x7f66c019c410 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f66b0000c00 tx=0x7f66b000bab0 comp rx=0 tx=0).stop 2024-03-24T02:59:52.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.302+0000 7f66c5e4a700 1 -- 172.21.15.17:0/2385375255 shutdown_connections 2024-03-24T02:59:52.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.302+0000 7f66c5e4a700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f66a80a29a0 0x7f66a80a4e60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:52.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.302+0000 7f66c5e4a700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c0107280 0x7f66c01965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:52.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.302+0000 7f66c5e4a700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c0101370 0x7f66c019c410 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:52.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.302+0000 7f66c5e4a700 1 --2- 172.21.15.17:0/2385375255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66c0101d30 0x7f66c019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:52.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.302+0000 7f66c5e4a700 1 -- 172.21.15.17:0/2385375255 >> 172.21.15.17:0/2385375255 conn(0x7f66c0077ef0 msgr2=0x7f66c00ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:52.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.302+0000 7f66c5e4a700 1 -- 172.21.15.17:0/2385375255 shutdown_connections 2024-03-24T02:59:52.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:52.302+0000 7f66c5e4a700 1 -- 172.21.15.17:0/2385375255 wait complete. 2024-03-24T02:59:53.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:53 smithi017 ceph-mon[31926]: pgmap v332: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-24T02:59:53.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:53 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2385375255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-24T02:59:53.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:53 smithi188 ceph-mon[36206]: pgmap v332: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-24T02:59:53.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:53 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2385375255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-24T02:59:53.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:53 smithi019 ceph-mon[35944]: pgmap v332: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-24T02:59:53.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:53 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2385375255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-24T02:59:55.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.100+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2014513170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef9c107520 msgr2=0x7fef9c10f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:55.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.100+0000 7fefa0f2d700 1 --2- 172.21.15.17:0/2014513170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef9c107520 0x7fef9c10f0a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fef940099c0 tx=0x7fef94009cd0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.100+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2014513170 shutdown_connections 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.100+0000 7fefa0f2d700 1 --2- 172.21.15.17:0/2014513170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef9c107520 0x7fef9c10f0a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.100+0000 7fefa0f2d700 1 --2- 172.21.15.17:0/2014513170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9c102470 0x7fef9c102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.100+0000 7fefa0f2d700 1 --2- 172.21.15.17:0/2014513170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c102e20 0x7fef9c106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.100+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2014513170 >> 172.21.15.17:0/2014513170 conn(0x7fef9c0792b0 msgr2=0x7fef9c0796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.100+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2014513170 shutdown_connections 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.100+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2014513170 wait complete. 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fefa0f2d700 1 Processor -- start 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fefa0f2d700 1 -- start start 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fefa0f2d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9c102470 0x7fef9c1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fefa0f2d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c102e20 0x7fef9c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fefa0f2d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef9c107520 0x7fef9c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fefa0f2d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1124e0 con 0x7fef9c107520 2024-03-24T02:59:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fefa0f2d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c112360 con 0x7fef9c102e20 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fefa0f2d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c112660 con 0x7fef9c102470 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fef9ad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef9c107520 0x7fef9c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fef9a59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9c102470 0x7fef9c1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fef9ad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef9c107520 0x7fef9c19a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54094/0 (socket says 172.21.15.17:54094) 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fef9ad9d700 1 -- 172.21.15.17:0/2948016389 learned_addr learned my addr 172.21.15.17:0/2948016389 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fef99d9b700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c102e20 0x7fef9c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fef99d9b700 1 -- 172.21.15.17:0/2948016389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9c102470 msgr2=0x7fef9c1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fef99d9b700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9c102470 0x7fef9c1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fef99d9b700 1 -- 172.21.15.17:0/2948016389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef9c107520 msgr2=0x7fef9c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fef99d9b700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef9c107520 0x7fef9c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fef99d9b700 1 -- 172.21.15.17:0/2948016389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c19b080 con 0x7fef9c102e20 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fef9ad9d700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef9c107520 0x7fef9c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.101+0000 7fef99d9b700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c102e20 0x7fef9c1a0ca0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef9c103620 tx=0x7fef8400eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.102+0000 7fef937fe700 1 -- 172.21.15.17:0/2948016389 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84010900 con 0x7fef9c102e20 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.102+0000 7fef937fe700 1 -- 172.21.15.17:0/2948016389 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef8400c430 con 0x7fef9c102e20 2024-03-24T02:59:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.102+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2948016389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c19b370 con 0x7fef9c102e20 2024-03-24T02:59:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.102+0000 7fef937fe700 1 -- 172.21.15.17:0/2948016389 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84018660 con 0x7fef9c102e20 2024-03-24T02:59:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.102+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2948016389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c1a7630 con 0x7fef9c102e20 2024-03-24T02:59:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.103+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2948016389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c068e50 con 0x7fef9c102e20 2024-03-24T02:59:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.104+0000 7fef937fe700 1 -- 172.21.15.17:0/2948016389 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fef8400c5e0 con 0x7fef9c102e20 2024-03-24T02:59:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.104+0000 7fef937fe700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fef880a2920 0x7fef880a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.104+0000 7fef9a59c700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fef880a2920 0x7fef880a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.104+0000 7fef937fe700 1 -- 172.21.15.17:0/2948016389 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fef840cf000 con 0x7fef9c102e20 2024-03-24T02:59:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.105+0000 7fef9a59c700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fef880a2920 0x7fef880a4de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef9c19ba60 tx=0x7fef8c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:55.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.109+0000 7fef937fe700 1 -- 172.21.15.17:0/2948016389 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef8409bc10 con 0x7fef9c102e20 2024-03-24T02:59:55.256 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:55 smithi017 ceph-mon[31926]: pgmap v333: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T02:59:55.256 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:55 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/390767284' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-24T02:59:55.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:55 smithi188 ceph-mon[36206]: pgmap v333: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T02:59:55.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:55 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/390767284' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-24T02:59:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.317+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2948016389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fef9c1a7860 con 0x7fef9c102e20 2024-03-24T02:59:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.317+0000 7fef937fe700 1 -- 172.21.15.17:0/2948016389 <== mon.2 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fef8409b510 con 0x7fef9c102e20 2024-03-24T02:59:55.316 INFO:teuthology.orchestra.run.smithi017.stdout:34359738452 2024-03-24T02:59:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.319+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2948016389 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fef880a2920 msgr2=0x7fef880a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.319+0000 7fefa0f2d700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fef880a2920 0x7fef880a4de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef9c19ba60 tx=0x7fef8c00b040 comp rx=0 tx=0).stop 2024-03-24T02:59:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.319+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2948016389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c102e20 msgr2=0x7fef9c1a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.319+0000 7fefa0f2d700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c102e20 0x7fef9c1a0ca0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef9c103620 tx=0x7fef8400eca0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.319+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2948016389 shutdown_connections 2024-03-24T02:59:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.319+0000 7fefa0f2d700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fef880a2920 0x7fef880a4de0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.319+0000 7fefa0f2d700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef9c107520 0x7fef9c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.319+0000 7fefa0f2d700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9c102470 0x7fef9c1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.319+0000 7fefa0f2d700 1 --2- 172.21.15.17:0/2948016389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c102e20 0x7fef9c1a0ca0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.319+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2948016389 >> 172.21.15.17:0/2948016389 conn(0x7fef9c0792b0 msgr2=0x7fef9c1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.320+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2948016389 shutdown_connections 2024-03-24T02:59:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.320+0000 7fefa0f2d700 1 -- 172.21.15.17:0/2948016389 wait complete. 2024-03-24T02:59:55.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:55 smithi019 ceph-mon[35944]: pgmap v333: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T02:59:55.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:55 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/390767284' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-24T02:59:55.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.364+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/2528431170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b58101320 msgr2=0x7f4b58101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:55.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.364+0000 7f4b5cd6b700 1 --2- 172.21.15.17:0/2528431170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b58101320 0x7f4b58101700 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009990 tx=0x7f4b48009ca0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.364+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/2528431170 shutdown_connections 2024-03-24T02:59:55.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.364+0000 7f4b5cd6b700 1 --2- 172.21.15.17:0/2528431170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b58106370 0x7f4b5810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.364+0000 7f4b5cd6b700 1 --2- 172.21.15.17:0/2528431170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b58101320 0x7f4b58101700 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.364+0000 7f4b5cd6b700 1 --2- 172.21.15.17:0/2528431170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b58101cd0 0x7f4b58105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.364+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/2528431170 >> 172.21.15.17:0/2528431170 conn(0x7f4b580770e0 msgr2=0x7f4b580774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:55.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.364+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/2528431170 shutdown_connections 2024-03-24T02:59:55.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.364+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/2528431170 wait complete. 2024-03-24T02:59:55.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.364+0000 7f4b5cd6b700 1 Processor -- start 2024-03-24T02:59:55.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b5cd6b700 1 -- start start 2024-03-24T02:59:55.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b5cd6b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b58101320 0x7f4b58102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:55.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b5cd6b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b58101cd0 0x7f4b58103270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b5cd6b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b58106370 0x7f4b581037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b5cd6b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b58111350 con 0x7f4b58101cd0 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b5cd6b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b581111d0 con 0x7f4b58101320 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b5cd6b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b581114d0 con 0x7f4b58106370 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b55d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b58101cd0 0x7f4b58103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b55d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b58101cd0 0x7f4b58103270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54106/0 (socket says 172.21.15.17:54106) 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b55d9b700 1 -- 172.21.15.17:0/3267901034 learned_addr learned my addr 172.21.15.17:0/3267901034 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b5659c700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b58101320 0x7f4b58102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b56d9d700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b58106370 0x7f4b581037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b55d9b700 1 -- 172.21.15.17:0/3267901034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b58106370 msgr2=0x7f4b581037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b55d9b700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b58106370 0x7f4b581037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b55d9b700 1 -- 172.21.15.17:0/3267901034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b58101320 msgr2=0x7f4b58102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b55d9b700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b58101320 0x7f4b58102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b55d9b700 1 -- 172.21.15.17:0/3267901034 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b581a6e30 con 0x7f4b58101cd0 2024-03-24T02:59:55.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.365+0000 7f4b55d9b700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b58101cd0 0x7f4b58103270 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00e970 tx=0x7f4b4c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:55.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.366+0000 7f4b477fe700 1 -- 172.21.15.17:0/3267901034 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c004a30 con 0x7f4b58101cd0 2024-03-24T02:59:55.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.366+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/3267901034 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b581a7120 con 0x7f4b58101cd0 2024-03-24T02:59:55.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.366+0000 7f4b477fe700 1 -- 172.21.15.17:0/3267901034 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b4c00cca0 con 0x7f4b58101cd0 2024-03-24T02:59:55.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.366+0000 7f4b477fe700 1 -- 172.21.15.17:0/3267901034 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c005680 con 0x7f4b58101cd0 2024-03-24T02:59:55.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.366+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/3267901034 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b581a76b0 con 0x7f4b58101cd0 2024-03-24T02:59:55.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.367+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/3267901034 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b58068e50 con 0x7f4b58101cd0 2024-03-24T02:59:55.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.368+0000 7f4b477fe700 1 -- 172.21.15.17:0/3267901034 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f4b4c00c7c0 con 0x7f4b58101cd0 2024-03-24T02:59:55.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.368+0000 7f4b477fe700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4b400a2920 0x7f4b400a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:55.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.369+0000 7f4b477fe700 1 -- 172.21.15.17:0/3267901034 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f4b4c0cf0b0 con 0x7f4b58101cd0 2024-03-24T02:59:55.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.369+0000 7f4b5659c700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4b400a2920 0x7f4b400a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:55.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.369+0000 7f4b5659c700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4b400a2920 0x7f4b400a4de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b480098e0 tx=0x7f4b48005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:55.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.373+0000 7f4b477fe700 1 -- 172.21.15.17:0/3267901034 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4b4c09bcc0 con 0x7f4b58101cd0 2024-03-24T02:59:55.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.583+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/3267901034 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4b5802da30 con 0x7f4b58101cd0 2024-03-24T02:59:55.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.583+0000 7f4b477fe700 1 -- 172.21.15.17:0/3267901034 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b4c09b5c0 con 0x7f4b58101cd0 2024-03-24T02:59:55.583 INFO:teuthology.orchestra.run.smithi017.stdout:55834574928 2024-03-24T02:59:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.586+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/3267901034 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4b400a2920 msgr2=0x7f4b400a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.586+0000 7f4b5cd6b700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4b400a2920 0x7f4b400a4de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b480098e0 tx=0x7f4b48005960 comp rx=0 tx=0).stop 2024-03-24T02:59:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.586+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/3267901034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b58101cd0 msgr2=0x7f4b58103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.586+0000 7f4b5cd6b700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b58101cd0 0x7f4b58103270 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00e970 tx=0x7f4b4c00ec80 comp rx=0 tx=0).stop 2024-03-24T02:59:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.586+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/3267901034 shutdown_connections 2024-03-24T02:59:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.586+0000 7f4b5cd6b700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4b400a2920 0x7f4b400a4de0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.586+0000 7f4b5cd6b700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b58106370 0x7f4b581037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.586+0000 7f4b5cd6b700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b58101320 0x7f4b58102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.586+0000 7f4b5cd6b700 1 --2- 172.21.15.17:0/3267901034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b58101cd0 0x7f4b58103270 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:55.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.586+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/3267901034 >> 172.21.15.17:0/3267901034 conn(0x7f4b580770e0 msgr2=0x7f4b58105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:55.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.586+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/3267901034 shutdown_connections 2024-03-24T02:59:55.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:55.586+0000 7f4b5cd6b700 1 -- 172.21.15.17:0/3267901034 wait complete. 2024-03-24T02:59:56.092 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738452 for osd.0 2024-03-24T02:59:56.093 DEBUG:teuthology.parallel:result is None 2024-03-24T02:59:56.292 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:56 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2948016389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T02:59:56.292 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:56 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3267901034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T02:59:56.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:56 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2948016389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T02:59:56.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:56 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3267901034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T02:59:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:56 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2948016389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T02:59:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:56 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3267901034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T02:59:56.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.513+0000 7f437e04c700 1 -- 172.21.15.17:0/3125361683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f437806b230 msgr2=0x7f437810f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:56.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.513+0000 7f437e04c700 1 --2- 172.21.15.17:0/3125361683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f437806b230 0x7f437810f050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f437000b0d0 tx=0x7f437000b3e0 comp rx=0 tx=0).stop 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.513+0000 7f437e04c700 1 -- 172.21.15.17:0/3125361683 shutdown_connections 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.513+0000 7f437e04c700 1 --2- 172.21.15.17:0/3125361683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f437806b230 0x7f437810f050 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.513+0000 7f437e04c700 1 --2- 172.21.15.17:0/3125361683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4378108210 0x7f43781085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.513+0000 7f437e04c700 1 --2- 172.21.15.17:0/3125361683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f437806a890 0x7f437806acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.513+0000 7f437e04c700 1 -- 172.21.15.17:0/3125361683 >> 172.21.15.17:0/3125361683 conn(0x7f43780770b0 msgr2=0x7f43780774c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.513+0000 7f437e04c700 1 -- 172.21.15.17:0/3125361683 shutdown_connections 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.513+0000 7f437e04c700 1 -- 172.21.15.17:0/3125361683 wait complete. 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.513+0000 7f437e04c700 1 Processor -- start 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.513+0000 7f437e04c700 1 -- start start 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f437e04c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f437806a890 0x7f4378196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f437e04c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f437806b230 0x7f4378196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f437e04c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378108210 0x7f437819b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:56.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f437e04c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4378110300 con 0x7f4378108210 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f437e04c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4378110180 con 0x7f437806b230 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f437e04c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4378110480 con 0x7f437806a890 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f4377fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378108210 0x7f437819b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f4377fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378108210 0x7f437819b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54124/0 (socket says 172.21.15.17:54124) 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f4377fff700 1 -- 172.21.15.17:0/413198049 learned_addr learned my addr 172.21.15.17:0/413198049 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f43777fe700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f437806a890 0x7f4378196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f4376ffd700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f437806b230 0x7f4378196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f4377fff700 1 -- 172.21.15.17:0/413198049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f437806a890 msgr2=0x7f4378196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f4377fff700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f437806a890 0x7f4378196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f4377fff700 1 -- 172.21.15.17:0/413198049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f437806b230 msgr2=0x7f4378196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f4377fff700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f437806b230 0x7f4378196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f4377fff700 1 -- 172.21.15.17:0/413198049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437819b740 con 0x7f4378108210 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f4377fff700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378108210 0x7f437819b000 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4370006070 tx=0x7f4370004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.514+0000 7f4374ff9700 1 -- 172.21.15.17:0/413198049 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370021070 con 0x7f4378108210 2024-03-24T02:59:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.515+0000 7f437e04c700 1 -- 172.21.15.17:0/413198049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437819b9d0 con 0x7f4378108210 2024-03-24T02:59:56.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.515+0000 7f4374ff9700 1 -- 172.21.15.17:0/413198049 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4370007b80 con 0x7f4378108210 2024-03-24T02:59:56.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.515+0000 7f4374ff9700 1 -- 172.21.15.17:0/413198049 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43700052a0 con 0x7f4378108210 2024-03-24T02:59:56.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.515+0000 7f437e04c700 1 -- 172.21.15.17:0/413198049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43781a32b0 con 0x7f4378108210 2024-03-24T02:59:56.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.515+0000 7f437e04c700 1 -- 172.21.15.17:0/413198049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437810a620 con 0x7f4378108210 2024-03-24T02:59:56.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.517+0000 7f4374ff9700 1 -- 172.21.15.17:0/413198049 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f4370024070 con 0x7f4378108210 2024-03-24T02:59:56.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.517+0000 7f4374ff9700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f43600a2920 0x7f43600a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:56.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.517+0000 7f43777fe700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f43600a2920 0x7f43600a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:56.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.517+0000 7f4374ff9700 1 -- 172.21.15.17:0/413198049 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f437000f020 con 0x7f4378108210 2024-03-24T02:59:56.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.518+0000 7f43777fe700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f43600a2920 0x7f43600a4de0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4368005950 tx=0x7f436800b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:56.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.523+0000 7f4374ff9700 1 -- 172.21.15.17:0/413198049 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f437002b020 con 0x7f4378108210 2024-03-24T02:59:56.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.733+0000 7f437e04c700 1 -- 172.21.15.17:0/413198049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4378197a90 con 0x7f4378108210 2024-03-24T02:59:56.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.733+0000 7f4374ff9700 1 -- 172.21.15.17:0/413198049 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f437009d560 con 0x7f4378108210 2024-03-24T02:59:56.733 INFO:teuthology.orchestra.run.smithi017.stdout:77309411404 2024-03-24T02:59:56.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.735+0000 7f437e04c700 1 -- 172.21.15.17:0/413198049 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f43600a2920 msgr2=0x7f43600a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:56.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.735+0000 7f437e04c700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f43600a2920 0x7f43600a4de0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4368005950 tx=0x7f436800b700 comp rx=0 tx=0).stop 2024-03-24T02:59:56.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.735+0000 7f437e04c700 1 -- 172.21.15.17:0/413198049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378108210 msgr2=0x7f437819b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:56.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.735+0000 7f437e04c700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378108210 0x7f437819b000 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4370006070 tx=0x7f4370004d00 comp rx=0 tx=0).stop 2024-03-24T02:59:56.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.735+0000 7f437e04c700 1 -- 172.21.15.17:0/413198049 shutdown_connections 2024-03-24T02:59:56.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.735+0000 7f437e04c700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f43600a2920 0x7f43600a4de0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:56.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.735+0000 7f437e04c700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378108210 0x7f437819b000 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:56.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.735+0000 7f437e04c700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f437806a890 0x7f4378196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:56.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.735+0000 7f437e04c700 1 --2- 172.21.15.17:0/413198049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f437806b230 0x7f4378196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:56.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.735+0000 7f437e04c700 1 -- 172.21.15.17:0/413198049 >> 172.21.15.17:0/413198049 conn(0x7f43780770b0 msgr2=0x7f43780ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:56.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.736+0000 7f437e04c700 1 -- 172.21.15.17:0/413198049 shutdown_connections 2024-03-24T02:59:56.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:56.736+0000 7f437e04c700 1 -- 172.21.15.17:0/413198049 wait complete. 2024-03-24T02:59:57.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:57 smithi188 ceph-mon[36206]: pgmap v334: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T02:59:57.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:57 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/413198049' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T02:59:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:57 smithi019 ceph-mon[35944]: pgmap v334: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T02:59:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:57 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/413198049' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T02:59:57.451 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:57 smithi017 ceph-mon[31926]: pgmap v334: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T02:59:57.451 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:57 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/413198049' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T02:59:57.478 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808867 got 287762808872 for osd.9 2024-03-24T02:59:57.478 DEBUG:teuthology.parallel:result is None 2024-03-24T02:59:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.866+0000 7fac1c548700 1 -- 172.21.15.17:0/3556282161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1406a7e0 msgr2=0x7fac1410f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.866+0000 7fac1c548700 1 --2- 172.21.15.17:0/3556282161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1406a7e0 0x7fac1410f020 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00b0d0 tx=0x7fac0c00b3e0 comp rx=0 tx=0).stop 2024-03-24T02:59:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.866+0000 7fac1c548700 1 -- 172.21.15.17:0/3556282161 shutdown_connections 2024-03-24T02:59:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.866+0000 7fac1c548700 1 --2- 172.21.15.17:0/3556282161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1406a7e0 0x7fac1410f020 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.866+0000 7fac1c548700 1 --2- 172.21.15.17:0/3556282161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac141081e0 0x7fac141085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.866+0000 7fac1c548700 1 --2- 172.21.15.17:0/3556282161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac14069e40 0x7fac1406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.866+0000 7fac1c548700 1 -- 172.21.15.17:0/3556282161 >> 172.21.15.17:0/3556282161 conn(0x7fac14077fc0 msgr2=0x7fac140783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.866+0000 7fac1c548700 1 -- 172.21.15.17:0/3556282161 shutdown_connections 2024-03-24T02:59:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.866+0000 7fac1c548700 1 -- 172.21.15.17:0/3556282161 wait complete. 2024-03-24T02:59:57.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac1c548700 1 Processor -- start 2024-03-24T02:59:57.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac1c548700 1 -- start start 2024-03-24T02:59:57.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac1c548700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac14069e40 0x7fac1419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:57.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac1c548700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac1406a7e0 0x7fac1419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:57.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac1c548700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac141081e0 0x7fac14198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:57.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac1c548700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac14112430 con 0x7fac14069e40 2024-03-24T02:59:57.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac1c548700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac141122b0 con 0x7fac141081e0 2024-03-24T02:59:57.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac1a2e4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac14069e40 0x7fac1419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:57.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac1c548700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac141125b0 con 0x7fac1406a7e0 2024-03-24T02:59:57.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac1a2e4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac14069e40 0x7fac1419e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54130/0 (socket says 172.21.15.17:54130) 2024-03-24T02:59:57.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac1a2e4700 1 -- 172.21.15.17:0/3222040486 learned_addr learned my addr 172.21.15.17:0/3222040486 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T02:59:57.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac19ae3700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac1406a7e0 0x7fac1419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:57.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.867+0000 7fac1aae5700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac141081e0 0x7fac14198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:57.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.868+0000 7fac1a2e4700 1 -- 172.21.15.17:0/3222040486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac1406a7e0 msgr2=0x7fac1419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:57.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.868+0000 7fac1a2e4700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac1406a7e0 0x7fac1419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:57.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.868+0000 7fac1a2e4700 1 -- 172.21.15.17:0/3222040486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac141081e0 msgr2=0x7fac14198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:57.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.868+0000 7fac1a2e4700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac141081e0 0x7fac14198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:57.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.868+0000 7fac1a2e4700 1 -- 172.21.15.17:0/3222040486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14198f50 con 0x7fac14069e40 2024-03-24T02:59:57.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.868+0000 7fac1a2e4700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac14069e40 0x7fac1419e5a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fac0400d980 tx=0x7fac0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:57.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.868+0000 7fac0b7fe700 1 -- 172.21.15.17:0/3222040486 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04016070 con 0x7fac14069e40 2024-03-24T02:59:57.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.868+0000 7fac1c548700 1 -- 172.21.15.17:0/3222040486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac14199240 con 0x7fac14069e40 2024-03-24T02:59:57.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.868+0000 7fac0b7fe700 1 -- 172.21.15.17:0/3222040486 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac04004d10 con 0x7fac14069e40 2024-03-24T02:59:57.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.868+0000 7fac0b7fe700 1 -- 172.21.15.17:0/3222040486 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04005670 con 0x7fac14069e40 2024-03-24T02:59:57.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.868+0000 7fac1c548700 1 -- 172.21.15.17:0/3222040486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac141a5520 con 0x7fac14069e40 2024-03-24T02:59:57.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.869+0000 7fac1c548700 1 -- 172.21.15.17:0/3222040486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac14103890 con 0x7fac14069e40 2024-03-24T02:59:57.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.870+0000 7fac0b7fe700 1 -- 172.21.15.17:0/3222040486 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fac04004750 con 0x7fac14069e40 2024-03-24T02:59:57.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.871+0000 7fac0b7fe700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fac000a2920 0x7fac000a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T02:59:57.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.871+0000 7fac0b7fe700 1 -- 172.21.15.17:0/3222040486 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fac040cf190 con 0x7fac14069e40 2024-03-24T02:59:57.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.871+0000 7fac19ae3700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fac000a2920 0x7fac000a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T02:59:57.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.871+0000 7fac19ae3700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fac000a2920 0x7fac000a4de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac14199e30 tx=0x7fac10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T02:59:57.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:57.875+0000 7fac0b7fe700 1 -- 172.21.15.17:0/3222040486 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fac0401f020 con 0x7fac14069e40 2024-03-24T02:59:58.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.085+0000 7fac1c548700 1 -- 172.21.15.17:0/3222040486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fac14199470 con 0x7fac14069e40 2024-03-24T02:59:58.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.086+0000 7fac0b7fe700 1 -- 172.21.15.17:0/3222040486 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fac040cce20 con 0x7fac14069e40 2024-03-24T02:59:58.085 INFO:teuthology.orchestra.run.smithi017.stdout:197568495672 2024-03-24T02:59:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.088+0000 7fac1c548700 1 -- 172.21.15.17:0/3222040486 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fac000a2920 msgr2=0x7fac000a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.088+0000 7fac1c548700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fac000a2920 0x7fac000a4de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac14199e30 tx=0x7fac10009260 comp rx=0 tx=0).stop 2024-03-24T02:59:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.088+0000 7fac1c548700 1 -- 172.21.15.17:0/3222040486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac14069e40 msgr2=0x7fac1419e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T02:59:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.088+0000 7fac1c548700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac14069e40 0x7fac1419e5a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fac0400d980 tx=0x7fac0400dc90 comp rx=0 tx=0).stop 2024-03-24T02:59:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.088+0000 7fac1c548700 1 -- 172.21.15.17:0/3222040486 shutdown_connections 2024-03-24T02:59:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.088+0000 7fac1c548700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fac000a2920 0x7fac000a4de0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.088+0000 7fac1c548700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac141081e0 0x7fac14198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.088+0000 7fac1c548700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac14069e40 0x7fac1419e5a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.088+0000 7fac1c548700 1 --2- 172.21.15.17:0/3222040486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac1406a7e0 0x7fac1419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T02:59:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.088+0000 7fac1c548700 1 -- 172.21.15.17:0/3222040486 >> 172.21.15.17:0/3222040486 conn(0x7fac14077fc0 msgr2=0x7fac140ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T02:59:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.088+0000 7fac1c548700 1 -- 172.21.15.17:0/3222040486 shutdown_connections 2024-03-24T02:59:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:58.088+0000 7fac1c548700 1 -- 172.21.15.17:0/3222040486 wait complete. 2024-03-24T02:59:59.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:59 smithi017 ceph-mon[31926]: pgmap v335: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T02:59:59.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 02:59:59 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3222040486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-24T02:59:59.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:59 smithi188 ceph-mon[36206]: pgmap v335: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T02:59:59.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 02:59:59 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3222040486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-24T02:59:59.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:59 smithi019 ceph-mon[35944]: pgmap v335: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T02:59:59.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 02:59:59 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3222040486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-24T02:59:59.394 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495668 got 197568495672 for osd.6 2024-03-24T02:59:59.394 DEBUG:teuthology.parallel:result is None 2024-03-24T03:00:00.129 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:00 smithi017 ceph-mon[31926]: overall HEALTH_OK 2024-03-24T03:00:00.160 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574928 for osd.1 2024-03-24T03:00:00.161 DEBUG:teuthology.parallel:result is None 2024-03-24T03:00:00.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:00 smithi188 ceph-mon[36206]: overall HEALTH_OK 2024-03-24T03:00:00.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:00 smithi019 ceph-mon[35944]: overall HEALTH_OK 2024-03-24T03:00:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.596+0000 7fad8081e700 1 -- 172.21.15.17:0/2151698074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad7c1070d0 msgr2=0x7fad7c1074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.596+0000 7fad8081e700 1 --2- 172.21.15.17:0/2151698074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad7c1070d0 0x7fad7c1074b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009990 tx=0x7fad6c009ca0 comp rx=0 tx=0).stop 2024-03-24T03:00:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.596+0000 7fad8081e700 1 -- 172.21.15.17:0/2151698074 shutdown_connections 2024-03-24T03:00:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.596+0000 7fad8081e700 1 --2- 172.21.15.17:0/2151698074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c101a30 0x7fad7c10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.596+0000 7fad8081e700 1 --2- 172.21.15.17:0/2151698074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad7c1070d0 0x7fad7c1074b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.596+0000 7fad8081e700 1 --2- 172.21.15.17:0/2151698074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c101090 0x7fad7c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.596+0000 7fad8081e700 1 -- 172.21.15.17:0/2151698074 >> 172.21.15.17:0/2151698074 conn(0x7fad7c077fc0 msgr2=0x7fad7c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.596+0000 7fad8081e700 1 -- 172.21.15.17:0/2151698074 shutdown_connections 2024-03-24T03:00:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.596+0000 7fad8081e700 1 -- 172.21.15.17:0/2151698074 wait complete. 2024-03-24T03:00:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad8081e700 1 Processor -- start 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad8081e700 1 -- start start 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad8081e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c101090 0x7fad7c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad8081e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad7c101a30 0x7fad7c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad8081e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c1070d0 0x7fad7c1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad8081e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c111310 con 0x7fad7c101a30 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad8081e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c111190 con 0x7fad7c1070d0 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad8081e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c111490 con 0x7fad7c101090 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad79d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad7c101a30 0x7fad7c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad79d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad7c101a30 0x7fad7c19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54078/0 (socket says 172.21.15.17:54078) 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad79d9b700 1 -- 172.21.15.17:0/390767284 learned_addr learned my addr 172.21.15.17:0/390767284 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad7a59c700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c101090 0x7fad7c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad7ad9d700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c1070d0 0x7fad7c1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad79d9b700 1 -- 172.21.15.17:0/390767284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c101090 msgr2=0x7fad7c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad79d9b700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c101090 0x7fad7c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad79d9b700 1 -- 172.21.15.17:0/390767284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c1070d0 msgr2=0x7fad7c1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad79d9b700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c1070d0 0x7fad7c1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad79d9b700 1 -- 172.21.15.17:0/390767284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c197d50 con 0x7fad7c101a30 2024-03-24T03:00:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad7a59c700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c101090 0x7fad7c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.597+0000 7fad79d9b700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad7c101a30 0x7fad7c19d970 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fad7000e9a0 tx=0x7fad7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.598+0000 7fad6b7fe700 1 -- 172.21.15.17:0/390767284 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000ca10 con 0x7fad7c101a30 2024-03-24T03:00:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.598+0000 7fad6b7fe700 1 -- 172.21.15.17:0/390767284 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad7000cbb0 con 0x7fad7c101a30 2024-03-24T03:00:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.598+0000 7fad8081e700 1 -- 172.21.15.17:0/390767284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c198040 con 0x7fad7c101a30 2024-03-24T03:00:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.598+0000 7fad6b7fe700 1 -- 172.21.15.17:0/390767284 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70018680 con 0x7fad7c101a30 2024-03-24T03:00:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.598+0000 7fad8081e700 1 -- 172.21.15.17:0/390767284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c1a4300 con 0x7fad7c101a30 2024-03-24T03:00:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.599+0000 7fad8081e700 1 -- 172.21.15.17:0/390767284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7c068e50 con 0x7fad7c101a30 2024-03-24T03:00:01.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.600+0000 7fad6b7fe700 1 -- 172.21.15.17:0/390767284 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fad70010430 con 0x7fad7c101a30 2024-03-24T03:00:01.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.600+0000 7fad6b7fe700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fad640a2920 0x7fad640a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:01.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.600+0000 7fad7a59c700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fad640a2920 0x7fad640a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:01.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.600+0000 7fad6b7fe700 1 -- 172.21.15.17:0/390767284 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fad700cf5f0 con 0x7fad7c101a30 2024-03-24T03:00:01.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.601+0000 7fad7a59c700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fad640a2920 0x7fad640a4de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b2e0 tx=0x7fad6c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:01.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.604+0000 7fad6b7fe700 1 -- 172.21.15.17:0/390767284 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fad7009c200 con 0x7fad7c101a30 2024-03-24T03:00:01.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.823+0000 7fad8081e700 1 -- 172.21.15.17:0/390767284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fad7c02da30 con 0x7fad7c101a30 2024-03-24T03:00:01.241 INFO:teuthology.orchestra.run.smithi017.stdout:253403070509 2024-03-24T03:00:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.823+0000 7fad6b7fe700 1 -- 172.21.15.17:0/390767284 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad7009bb00 con 0x7fad7c101a30 2024-03-24T03:00:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.825+0000 7fad8081e700 1 -- 172.21.15.17:0/390767284 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fad640a2920 msgr2=0x7fad640a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.825+0000 7fad8081e700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fad640a2920 0x7fad640a4de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b2e0 tx=0x7fad6c005960 comp rx=0 tx=0).stop 2024-03-24T03:00:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.825+0000 7fad8081e700 1 -- 172.21.15.17:0/390767284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad7c101a30 msgr2=0x7fad7c19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.825+0000 7fad8081e700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad7c101a30 0x7fad7c19d970 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fad7000e9a0 tx=0x7fad7000ecb0 comp rx=0 tx=0).stop 2024-03-24T03:00:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.825+0000 7fad8081e700 1 -- 172.21.15.17:0/390767284 shutdown_connections 2024-03-24T03:00:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.825+0000 7fad8081e700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fad640a2920 0x7fad640a4de0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.825+0000 7fad8081e700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c1070d0 0x7fad7c1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.825+0000 7fad8081e700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c101090 0x7fad7c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.825+0000 7fad8081e700 1 --2- 172.21.15.17:0/390767284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad7c101a30 0x7fad7c19d970 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.825+0000 7fad8081e700 1 -- 172.21.15.17:0/390767284 >> 172.21.15.17:0/390767284 conn(0x7fad7c077fc0 msgr2=0x7fad7c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.826+0000 7fad8081e700 1 -- 172.21.15.17:0/390767284 shutdown_connections 2024-03-24T03:00:01.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T02:59:54.826+0000 7fad8081e700 1 -- 172.21.15.17:0/390767284 wait complete. 2024-03-24T03:00:01.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:01 smithi019 ceph-mon[35944]: pgmap v336: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-24T03:00:01.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:01 smithi017 ceph-mon[31926]: pgmap v336: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-24T03:00:01.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:01 smithi188 ceph-mon[36206]: pgmap v336: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-24T03:00:01.859 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070505 got 253403070509 for osd.8 2024-03-24T03:00:01.859 DEBUG:teuthology.parallel:result is None 2024-03-24T03:00:02.093 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:02 smithi017 ceph-mon[31926]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-24T03:00:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.094+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/1853231167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd41081c0 msgr2=0x7f1dd41085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.094+0000 7f1dd9f7c700 1 --2- 172.21.15.17:0/1853231167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd41081c0 0x7f1dd41085a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40099c0 tx=0x7f1dc4009cd0 comp rx=0 tx=0).stop 2024-03-24T03:00:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.094+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/1853231167 shutdown_connections 2024-03-24T03:00:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.094+0000 7f1dd9f7c700 1 --2- 172.21.15.17:0/1853231167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dd4102ba0 0x7f1dd410efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.094+0000 7f1dd9f7c700 1 --2- 172.21.15.17:0/1853231167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd41081c0 0x7f1dd41085a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.094+0000 7f1dd9f7c700 1 --2- 172.21.15.17:0/1853231167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd41021e0 0x7f1dd4102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.095+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/1853231167 >> 172.21.15.17:0/1853231167 conn(0x7f1dd40792c0 msgr2=0x7f1dd40796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:02.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.095+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/1853231167 shutdown_connections 2024-03-24T03:00:02.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.095+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/1853231167 wait complete. 2024-03-24T03:00:02.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.095+0000 7f1dd9f7c700 1 Processor -- start 2024-03-24T03:00:02.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd9f7c700 1 -- start start 2024-03-24T03:00:02.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd9f7c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd41021e0 0x7f1dd419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:02.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd9f7c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dd4102ba0 0x7f1dd419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd9f7c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd41081c0 0x7f1dd4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd9f7c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4112470 con 0x7f1dd41081c0 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd9f7c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd41122f0 con 0x7f1dd41021e0 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd9f7c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd41125f0 con 0x7f1dd4102ba0 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd41021e0 0x7f1dd419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dd4102ba0 0x7f1dd419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd41081c0 0x7f1dd4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd41021e0 0x7f1dd419e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:50190/0 (socket says 172.21.15.17:50190) 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dd4102ba0 0x7f1dd419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:47948/0 (socket says 172.21.15.17:47948) 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.096+0000 7f1dd37fe700 1 -- 172.21.15.17:0/3867721240 learned_addr learned my addr 172.21.15.17:0/3867721240 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd2ffd700 1 -- 172.21.15.17:0/3867721240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd41021e0 msgr2=0x7f1dd419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd2ffd700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd41021e0 0x7f1dd419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd2ffd700 1 -- 172.21.15.17:0/3867721240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd41081c0 msgr2=0x7f1dd4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd2ffd700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd41081c0 0x7f1dd4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd2ffd700 1 -- 172.21.15.17:0/3867721240 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd4198f50 con 0x7f1dd4102ba0 2024-03-24T03:00:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd37fe700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd41021e0 0x7f1dd419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd3fff700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd41081c0 0x7f1dd4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd2ffd700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dd4102ba0 0x7f1dd419eae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800e9f0 tx=0x7f1dc800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd0ff9700 1 -- 172.21.15.17:0/3867721240 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc800c9c0 con 0x7f1dd4102ba0 2024-03-24T03:00:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd0ff9700 1 -- 172.21.15.17:0/3867721240 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dc800cb60 con 0x7f1dd4102ba0 2024-03-24T03:00:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/3867721240 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd4199240 con 0x7f1dd4102ba0 2024-03-24T03:00:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd0ff9700 1 -- 172.21.15.17:0/3867721240 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8018680 con 0x7f1dd4102ba0 2024-03-24T03:00:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.097+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/3867721240 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd41a5520 con 0x7f1dd4102ba0 2024-03-24T03:00:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.098+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/3867721240 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd4103870 con 0x7f1dd4102ba0 2024-03-24T03:00:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.099+0000 7f1dd0ff9700 1 -- 172.21.15.17:0/3867721240 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1dc8018820 con 0x7f1dd4102ba0 2024-03-24T03:00:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.099+0000 7f1dd0ff9700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1dbc0a2910 0x7f1dbc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.100+0000 7f1dd37fe700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1dbc0a2910 0x7f1dbc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.100+0000 7f1dd0ff9700 1 -- 172.21.15.17:0/3867721240 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f1dc80cee20 con 0x7f1dd4102ba0 2024-03-24T03:00:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.100+0000 7f1dd37fe700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1dbc0a2910 0x7f1dbc0a4dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009990 tx=0x7f1dc4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:02.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.104+0000 7f1dd0ff9700 1 -- 172.21.15.17:0/3867721240 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1dc809bb90 con 0x7f1dd4102ba0 2024-03-24T03:00:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.313+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/3867721240 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f1dd4199ba0 con 0x7f1dd4102ba0 2024-03-24T03:00:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.313+0000 7f1dd0ff9700 1 -- 172.21.15.17:0/3867721240 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f1dc809b490 con 0x7f1dd4102ba0 2024-03-24T03:00:02.313 INFO:teuthology.orchestra.run.smithi017.stdout:309237645348 2024-03-24T03:00:02.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.316+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/3867721240 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1dbc0a2910 msgr2=0x7f1dbc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:02.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.316+0000 7f1dd9f7c700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1dbc0a2910 0x7f1dbc0a4dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009990 tx=0x7f1dc4005960 comp rx=0 tx=0).stop 2024-03-24T03:00:02.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.316+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/3867721240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dd4102ba0 msgr2=0x7f1dd419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:02.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.316+0000 7f1dd9f7c700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dd4102ba0 0x7f1dd419eae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800e9f0 tx=0x7f1dc800ed00 comp rx=0 tx=0).stop 2024-03-24T03:00:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.316+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/3867721240 shutdown_connections 2024-03-24T03:00:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.316+0000 7f1dd9f7c700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1dbc0a2910 0x7f1dbc0a4dd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.316+0000 7f1dd9f7c700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd41081c0 0x7f1dd4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.316+0000 7f1dd9f7c700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd41021e0 0x7f1dd419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.316+0000 7f1dd9f7c700 1 --2- 172.21.15.17:0/3867721240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dd4102ba0 0x7f1dd419eae0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.316+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/3867721240 >> 172.21.15.17:0/3867721240 conn(0x7f1dd40792c0 msgr2=0x7f1dd4100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.316+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/3867721240 shutdown_connections 2024-03-24T03:00:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:02.316+0000 7f1dd9f7c700 1 -- 172.21.15.17:0/3867721240 wait complete. 2024-03-24T03:00:02.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:02 smithi019 ceph-mon[35944]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-24T03:00:02.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:02 smithi188 ceph-mon[36206]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-24T03:00:03.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:03 smithi019 ceph-mon[35944]: pgmap v337: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-24T03:00:03.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:03 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3867721240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-24T03:00:03.387 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:03 smithi017 ceph-mon[31926]: pgmap v337: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-24T03:00:03.387 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:03 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3867721240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-24T03:00:03.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:03 smithi188 ceph-mon[36206]: pgmap v337: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-24T03:00:03.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:03 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3867721240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-24T03:00:03.755 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645343 got 309237645348 for osd.10 2024-03-24T03:00:03.755 DEBUG:teuthology.parallel:result is None 2024-03-24T03:00:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.449+0000 7f3d28bf8700 1 -- 172.21.15.17:0/1915091072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24105800 msgr2=0x7f3d24105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.450+0000 7f3d28bf8700 1 --2- 172.21.15.17:0/1915091072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24105800 0x7f3d24105be0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009990 tx=0x7f3d14009ca0 comp rx=0 tx=0).stop 2024-03-24T03:00:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.450+0000 7f3d28bf8700 1 -- 172.21.15.17:0/1915091072 shutdown_connections 2024-03-24T03:00:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.450+0000 7f3d28bf8700 1 --2- 172.21.15.17:0/1915091072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d24101500 0x7f3d2410d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.450+0000 7f3d28bf8700 1 --2- 172.21.15.17:0/1915091072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24105800 0x7f3d24105be0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.450+0000 7f3d28bf8700 1 --2- 172.21.15.17:0/1915091072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d24100b40 0x7f3d24100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.450+0000 7f3d28bf8700 1 -- 172.21.15.17:0/1915091072 >> 172.21.15.17:0/1915091072 conn(0x7f3d24077260 msgr2=0x7f3d240fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.450+0000 7f3d28bf8700 1 -- 172.21.15.17:0/1915091072 shutdown_connections 2024-03-24T03:00:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.450+0000 7f3d28bf8700 1 -- 172.21.15.17:0/1915091072 wait complete. 2024-03-24T03:00:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.450+0000 7f3d28bf8700 1 Processor -- start 2024-03-24T03:00:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.450+0000 7f3d28bf8700 1 -- start start 2024-03-24T03:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d28bf8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d24100b40 0x7f3d241a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d28bf8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24101500 0x7f3d241a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d28bf8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d24105800 0x7f3d2419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d28bf8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d24110260 con 0x7f3d24101500 2024-03-24T03:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d28bf8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241100e0 con 0x7f3d24100b40 2024-03-24T03:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d28bf8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241103e0 con 0x7f3d24105800 2024-03-24T03:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d21d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24101500 0x7f3d241a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d21d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24101500 0x7f3d241a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37426/0 (socket says 172.21.15.17:37426) 2024-03-24T03:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d21d9b700 1 -- 172.21.15.17:0/434123111 learned_addr learned my addr 172.21.15.17:0/434123111 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d2259c700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d24100b40 0x7f3d241a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d22d9d700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d24105800 0x7f3d2419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d21d9b700 1 -- 172.21.15.17:0/434123111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d24105800 msgr2=0x7f3d2419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d21d9b700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d24105800 0x7f3d2419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d21d9b700 1 -- 172.21.15.17:0/434123111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d24100b40 msgr2=0x7f3d241a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d21d9b700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d24100b40 0x7f3d241a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d21d9b700 1 -- 172.21.15.17:0/434123111 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2419b000 con 0x7f3d24101500 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d2259c700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d24100b40 0x7f3d241a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d21d9b700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24101500 0x7f3d241a0c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800ead0 tx=0x7f3d1800ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.451+0000 7f3d137fe700 1 -- 172.21.15.17:0/434123111 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800ccd0 con 0x7f3d24101500 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.452+0000 7f3d137fe700 1 -- 172.21.15.17:0/434123111 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d18004d10 con 0x7f3d24101500 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.452+0000 7f3d28bf8700 1 -- 172.21.15.17:0/434123111 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2419b2f0 con 0x7f3d24101500 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.452+0000 7f3d137fe700 1 -- 172.21.15.17:0/434123111 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180155f0 con 0x7f3d24101500 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.452+0000 7f3d28bf8700 1 -- 172.21.15.17:0/434123111 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d241a75b0 con 0x7f3d24101500 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.452+0000 7f3d28bf8700 1 -- 172.21.15.17:0/434123111 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d24068e50 con 0x7f3d24101500 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.454+0000 7f3d137fe700 1 -- 172.21.15.17:0/434123111 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3d18004ec0 con 0x7f3d24101500 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.454+0000 7f3d137fe700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3d0c0a2890 0x7f3d0c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.454+0000 7f3d2259c700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3d0c0a2890 0x7f3d0c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.454+0000 7f3d137fe700 1 -- 172.21.15.17:0/434123111 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f3d180cebf0 con 0x7f3d24101500 2024-03-24T03:00:04.876 INFO:teuthology.orchestra.run.smithi017.stdout:219043332147 2024-03-24T03:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.455+0000 7f3d2259c700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3d0c0a2890 0x7f3d0c0a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400b2e0 tx=0x7f3d14005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.459+0000 7f3d137fe700 1 -- 172.21.15.17:0/434123111 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3d1809b9f0 con 0x7f3d24101500 2024-03-24T03:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.668+0000 7f3d28bf8700 1 -- 172.21.15.17:0/434123111 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3d2419bdd0 con 0x7f3d24101500 2024-03-24T03:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.668+0000 7f3d137fe700 1 -- 172.21.15.17:0/434123111 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3d2419bdd0 con 0x7f3d24101500 2024-03-24T03:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.670+0000 7f3d28bf8700 1 -- 172.21.15.17:0/434123111 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3d0c0a2890 msgr2=0x7f3d0c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.670+0000 7f3d28bf8700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3d0c0a2890 0x7f3d0c0a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400b2e0 tx=0x7f3d14005960 comp rx=0 tx=0).stop 2024-03-24T03:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.670+0000 7f3d28bf8700 1 -- 172.21.15.17:0/434123111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24101500 msgr2=0x7f3d241a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.670+0000 7f3d28bf8700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24101500 0x7f3d241a0c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800ead0 tx=0x7f3d1800ede0 comp rx=0 tx=0).stop 2024-03-24T03:00:04.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.670+0000 7f3d28bf8700 1 -- 172.21.15.17:0/434123111 shutdown_connections 2024-03-24T03:00:04.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.670+0000 7f3d28bf8700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3d0c0a2890 0x7f3d0c0a4d50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:04.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.670+0000 7f3d28bf8700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d24105800 0x7f3d2419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:04.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.670+0000 7f3d28bf8700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d24100b40 0x7f3d241a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:04.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.670+0000 7f3d28bf8700 1 --2- 172.21.15.17:0/434123111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24101500 0x7f3d241a0c20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:04.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.670+0000 7f3d28bf8700 1 -- 172.21.15.17:0/434123111 >> 172.21.15.17:0/434123111 conn(0x7f3d24077260 msgr2=0x7f3d24103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:04.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.670+0000 7f3d28bf8700 1 -- 172.21.15.17:0/434123111 shutdown_connections 2024-03-24T03:00:04.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:04.670+0000 7f3d28bf8700 1 -- 172.21.15.17:0/434123111 wait complete. 2024-03-24T03:00:05.177 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:05 smithi017 ceph-mon[31926]: pgmap v338: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-24T03:00:05.177 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:05 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/434123111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-24T03:00:05.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:05 smithi019 ceph-mon[35944]: pgmap v338: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-24T03:00:05.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:05 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/434123111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-24T03:00:05.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:05 smithi188 ceph-mon[36206]: pgmap v338: 129 pgs: 129 active+clean; 649 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-24T03:00:05.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:05 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/434123111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-24T03:00:05.755 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332143 got 219043332147 for osd.7 2024-03-24T03:00:05.755 DEBUG:teuthology.parallel:result is None 2024-03-24T03:00:06.625 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411398 got 77309411404 for osd.2 2024-03-24T03:00:06.625 DEBUG:teuthology.parallel:result is None 2024-03-24T03:00:07.835 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:07 smithi017 ceph-mon[31926]: pgmap v339: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-24T03:00:07.835 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:07 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T03:00:07.835 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:07 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T03:00:08.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:07 smithi188 ceph-mon[36206]: pgmap v339: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-24T03:00:08.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:07 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T03:00:08.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:07 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T03:00:08.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:07 smithi019 ceph-mon[35944]: pgmap v339: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-24T03:00:08.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:07 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T03:00:08.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:07 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T03:00:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.075+0000 7f9650c45700 1 -- 172.21.15.17:0/879319131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964c107260 msgr2=0x7f964c107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.075+0000 7f9650c45700 1 --2- 172.21.15.17:0/879319131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964c107260 0x7f964c107640 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f963c0099c0 tx=0x7f963c009cd0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.076+0000 7f9650c45700 1 -- 172.21.15.17:0/879319131 shutdown_connections 2024-03-24T03:00:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.076+0000 7f9650c45700 1 --2- 172.21.15.17:0/879319131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c101550 0x7f964c10f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.076+0000 7f9650c45700 1 --2- 172.21.15.17:0/879319131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964c107260 0x7f964c107640 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.076+0000 7f9650c45700 1 --2- 172.21.15.17:0/879319131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c100b90 0x7f964c101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.076+0000 7f9650c45700 1 -- 172.21.15.17:0/879319131 >> 172.21.15.17:0/879319131 conn(0x7f964c0763e0 msgr2=0x7f964c0767f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.076+0000 7f9650c45700 1 -- 172.21.15.17:0/879319131 shutdown_connections 2024-03-24T03:00:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.076+0000 7f9650c45700 1 -- 172.21.15.17:0/879319131 wait complete. 2024-03-24T03:00:08.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.076+0000 7f9650c45700 1 Processor -- start 2024-03-24T03:00:08.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.076+0000 7f9650c45700 1 -- start start 2024-03-24T03:00:08.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f9650c45700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c100b90 0x7f964c101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:08.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f9650c45700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c101550 0x7f964c1022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:08.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f9650c45700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964c107260 0x7f964c1027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f9650c45700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c112510 con 0x7f964c107260 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f9650c45700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c112390 con 0x7f964c101550 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f9650c45700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c112690 con 0x7f964c100b90 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f964ad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964c107260 0x7f964c1027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f964a59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c100b90 0x7f964c101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f964ad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964c107260 0x7f964c1027f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37446/0 (socket says 172.21.15.17:37446) 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f964a59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c100b90 0x7f964c101d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:47984/0 (socket says 172.21.15.17:47984) 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f964ad9d700 1 -- 172.21.15.17:0/1627534258 learned_addr learned my addr 172.21.15.17:0/1627534258 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f9649d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c101550 0x7f964c1022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f964ad9d700 1 -- 172.21.15.17:0/1627534258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c100b90 msgr2=0x7f964c101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f964ad9d700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c100b90 0x7f964c101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f964ad9d700 1 -- 172.21.15.17:0/1627534258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c101550 msgr2=0x7f964c1022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f964ad9d700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c101550 0x7f964c1022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f964ad9d700 1 -- 172.21.15.17:0/1627534258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c1a6e50 con 0x7f964c107260 2024-03-24T03:00:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f9649d9b700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c101550 0x7f964c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f964a59c700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c100b90 0x7f964c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.077+0000 7f964ad9d700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964c107260 0x7f964c1027f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f964400ef30 tx=0x7f964400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.078+0000 7f963b7fe700 1 -- 172.21.15.17:0/1627534258 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96440092e0 con 0x7f964c107260 2024-03-24T03:00:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.078+0000 7f9650c45700 1 -- 172.21.15.17:0/1627534258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c1a70e0 con 0x7f964c107260 2024-03-24T03:00:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.078+0000 7f963b7fe700 1 -- 172.21.15.17:0/1627534258 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9644004bb0 con 0x7f964c107260 2024-03-24T03:00:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.078+0000 7f963b7fe700 1 -- 172.21.15.17:0/1627534258 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644004280 con 0x7f964c107260 2024-03-24T03:00:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.078+0000 7f9650c45700 1 -- 172.21.15.17:0/1627534258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c1a7620 con 0x7f964c107260 2024-03-24T03:00:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.079+0000 7f9650c45700 1 -- 172.21.15.17:0/1627534258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964c10a810 con 0x7f964c107260 2024-03-24T03:00:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.080+0000 7f963b7fe700 1 -- 172.21.15.17:0/1627534258 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f964401b030 con 0x7f964c107260 2024-03-24T03:00:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.080+0000 7f963b7fe700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96340a2890 0x7f96340a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.080+0000 7f964a59c700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96340a2890 0x7f96340a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.080+0000 7f963b7fe700 1 -- 172.21.15.17:0/1627534258 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f96440cf180 con 0x7f964c107260 2024-03-24T03:00:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.081+0000 7f964a59c700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96340a2890 0x7f96340a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f963c009990 tx=0x7f963c005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:08.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.084+0000 7f963b7fe700 1 -- 172.21.15.17:0/1627534258 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9644022020 con 0x7f964c107260 2024-03-24T03:00:08.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.291+0000 7f9650c45700 1 -- 172.21.15.17:0/1627534258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f964c1032a0 con 0x7f964c107260 2024-03-24T03:00:08.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.291+0000 7f963b7fe700 1 -- 172.21.15.17:0/1627534258 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9644010040 con 0x7f964c107260 2024-03-24T03:00:08.291 INFO:teuthology.orchestra.run.smithi017.stdout:163208757311 2024-03-24T03:00:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.293+0000 7f9650c45700 1 -- 172.21.15.17:0/1627534258 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96340a2890 msgr2=0x7f96340a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.293+0000 7f9650c45700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96340a2890 0x7f96340a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f963c009990 tx=0x7f963c005a90 comp rx=0 tx=0).stop 2024-03-24T03:00:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.293+0000 7f9650c45700 1 -- 172.21.15.17:0/1627534258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964c107260 msgr2=0x7f964c1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.293+0000 7f9650c45700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964c107260 0x7f964c1027f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f964400ef30 tx=0x7f964400c3a0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.294+0000 7f9650c45700 1 -- 172.21.15.17:0/1627534258 shutdown_connections 2024-03-24T03:00:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.294+0000 7f9650c45700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96340a2890 0x7f96340a4d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.294+0000 7f9650c45700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964c107260 0x7f964c1027f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.294+0000 7f9650c45700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c100b90 0x7f964c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.294+0000 7f9650c45700 1 --2- 172.21.15.17:0/1627534258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c101550 0x7f964c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.294+0000 7f9650c45700 1 -- 172.21.15.17:0/1627534258 >> 172.21.15.17:0/1627534258 conn(0x7f964c0763e0 msgr2=0x7f964c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:08.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.294+0000 7f9650c45700 1 -- 172.21.15.17:0/1627534258 shutdown_connections 2024-03-24T03:00:08.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.294+0000 7f9650c45700 1 -- 172.21.15.17:0/1627534258 wait complete. 2024-03-24T03:00:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.321+0000 7f1b0927d700 1 -- 172.21.15.17:0/4244579108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b04102bd0 msgr2=0x7f1b0410f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.321+0000 7f1b0927d700 1 --2- 172.21.15.17:0/4244579108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b04102bd0 0x7f1b0410f060 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00b0a0 tx=0x7f1afc00b3b0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.321+0000 7f1b0927d700 1 -- 172.21.15.17:0/4244579108 shutdown_connections 2024-03-24T03:00:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.321+0000 7f1b0927d700 1 --2- 172.21.15.17:0/4244579108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b04102bd0 0x7f1b0410f060 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.321+0000 7f1b0927d700 1 --2- 172.21.15.17:0/4244579108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b04108260 0x7f1b04108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.321+0000 7f1b0927d700 1 --2- 172.21.15.17:0/4244579108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b04102210 0x7f1b04102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.321+0000 7f1b0927d700 1 -- 172.21.15.17:0/4244579108 >> 172.21.15.17:0/4244579108 conn(0x7f1b040792e0 msgr2=0x7f1b040796f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.321+0000 7f1b0927d700 1 -- 172.21.15.17:0/4244579108 shutdown_connections 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.321+0000 7f1b0927d700 1 -- 172.21.15.17:0/4244579108 wait complete. 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.322+0000 7f1b0927d700 1 Processor -- start 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.322+0000 7f1b0927d700 1 -- start start 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.322+0000 7f1b0927d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b04102210 0x7f1b0419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b0927d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b04102bd0 0x7f1b0419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b0927d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b04108260 0x7f1b041986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b0927d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b041124a0 con 0x7f1b04102bd0 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b0927d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b04112320 con 0x7f1b04102210 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b0927d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b04112620 con 0x7f1b04108260 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b02ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b04102210 0x7f1b0419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b037fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b04108260 0x7f1b041986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:08.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b02ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b04102210 0x7f1b0419e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:50232/0 (socket says 172.21.15.17:50232) 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b02ffd700 1 -- 172.21.15.17:0/2466528660 learned_addr learned my addr 172.21.15.17:0/2466528660 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b027fc700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b04102bd0 0x7f1b0419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b037fe700 1 -- 172.21.15.17:0/2466528660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b04102210 msgr2=0x7f1b0419e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b037fe700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b04102210 0x7f1b0419e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b037fe700 1 -- 172.21.15.17:0/2466528660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b04102bd0 msgr2=0x7f1b0419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b037fe700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b04102bd0 0x7f1b0419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b037fe700 1 -- 172.21.15.17:0/2466528660 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b04198fc0 con 0x7f1b04108260 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b027fc700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b04102bd0 0x7f1b0419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.323+0000 7f1b037fe700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b04108260 0x7f1b041986e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009bc0 tx=0x7f1afc004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.324+0000 7f1aebfff700 1 -- 172.21.15.17:0/2466528660 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc021070 con 0x7f1b04108260 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.324+0000 7f1b0927d700 1 -- 172.21.15.17:0/2466528660 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b04199250 con 0x7f1b04108260 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.324+0000 7f1aebfff700 1 -- 172.21.15.17:0/2466528660 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1afc007b80 con 0x7f1b04108260 2024-03-24T03:00:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.324+0000 7f1aebfff700 1 -- 172.21.15.17:0/2466528660 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc005040 con 0x7f1b04108260 2024-03-24T03:00:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.324+0000 7f1b0927d700 1 -- 172.21.15.17:0/2466528660 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b041a5590 con 0x7f1b04108260 2024-03-24T03:00:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.325+0000 7f1b0927d700 1 -- 172.21.15.17:0/2466528660 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0410a640 con 0x7f1b04108260 2024-03-24T03:00:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.326+0000 7f1aebfff700 1 -- 172.21.15.17:0/2466528660 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1afc024070 con 0x7f1b04108260 2024-03-24T03:00:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.327+0000 7f1aebfff700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1aec0a2890 0x7f1aec0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.327+0000 7f1b02ffd700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1aec0a2890 0x7f1aec0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.327+0000 7f1aebfff700 1 -- 172.21.15.17:0/2466528660 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f1afc00f020 con 0x7f1b04108260 2024-03-24T03:00:08.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.327+0000 7f1b02ffd700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1aec0a2890 0x7f1aec0a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009680 tx=0x7f1af4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:08.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.330+0000 7f1aebfff700 1 -- 172.21.15.17:0/2466528660 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1afc02b020 con 0x7f1b04108260 2024-03-24T03:00:08.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.534+0000 7f1b0927d700 1 -- 172.21.15.17:0/2466528660 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1b04199c10 con 0x7f1b04108260 2024-03-24T03:00:08.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.534+0000 7f1aebfff700 1 -- 172.21.15.17:0/2466528660 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1afc011e50 con 0x7f1b04108260 2024-03-24T03:00:08.534 INFO:teuthology.orchestra.run.smithi017.stdout:107374182472 2024-03-24T03:00:08.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.537+0000 7f1b0927d700 1 -- 172.21.15.17:0/2466528660 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1aec0a2890 msgr2=0x7f1aec0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:08.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.537+0000 7f1b0927d700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1aec0a2890 0x7f1aec0a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009680 tx=0x7f1af4009260 comp rx=0 tx=0).stop 2024-03-24T03:00:08.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.537+0000 7f1b0927d700 1 -- 172.21.15.17:0/2466528660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b04108260 msgr2=0x7f1b041986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:08.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.537+0000 7f1b0927d700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b04108260 0x7f1b041986e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009bc0 tx=0x7f1afc004c80 comp rx=0 tx=0).stop 2024-03-24T03:00:08.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.537+0000 7f1b0927d700 1 -- 172.21.15.17:0/2466528660 shutdown_connections 2024-03-24T03:00:08.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.537+0000 7f1b0927d700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1aec0a2890 0x7f1aec0a4d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.537+0000 7f1b0927d700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b04108260 0x7f1b041986e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.537+0000 7f1b0927d700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b04102210 0x7f1b0419e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.537+0000 7f1b0927d700 1 --2- 172.21.15.17:0/2466528660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b04102bd0 0x7f1b0419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:08.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.537+0000 7f1b0927d700 1 -- 172.21.15.17:0/2466528660 >> 172.21.15.17:0/2466528660 conn(0x7f1b040792e0 msgr2=0x7f1b04101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:08.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.537+0000 7f1b0927d700 1 -- 172.21.15.17:0/2466528660 shutdown_connections 2024-03-24T03:00:08.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:08.537+0000 7f1b0927d700 1 -- 172.21.15.17:0/2466528660 wait complete. 2024-03-24T03:00:08.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:00:08.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:00:08.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:08 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T03:00:08.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:08 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1627534258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T03:00:08.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:08 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2466528660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T03:00:09.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:00:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:00:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:08 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T03:00:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:08 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1627534258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T03:00:09.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:08 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2466528660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T03:00:09.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:00:09.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:00:09.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:08 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' 2024-03-24T03:00:09.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:08 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1627534258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T03:00:09.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:08 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2466528660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T03:00:09.199 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757304 got 163208757311 for osd.5 2024-03-24T03:00:09.199 DEBUG:teuthology.parallel:result is None 2024-03-24T03:00:09.758 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182465 got 107374182472 for osd.3 2024-03-24T03:00:09.758 DEBUG:teuthology.parallel:result is None 2024-03-24T03:00:09.758 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-24T03:00:09.758 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-24T03:00:10.012 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:00:10.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:09 smithi188 ceph-mon[36206]: pgmap v340: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-24T03:00:10.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:09 smithi019 ceph-mon[35944]: pgmap v340: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-24T03:00:10.133 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:09 smithi017 ceph-mon[31926]: pgmap v340: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-24T03:00:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.750+0000 7f94e216e700 1 -- 172.21.15.17:0/89892105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94dc069e40 msgr2=0x7f94dc06a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.750+0000 7f94e216e700 1 --2- 172.21.15.17:0/89892105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94dc069e40 0x7f94dc06a2a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f94c40099c0 tx=0x7f94c4009cd0 comp rx=0 tx=0).stop 2024-03-24T03:00:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.750+0000 7f94e216e700 1 -- 172.21.15.17:0/89892105 shutdown_connections 2024-03-24T03:00:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.750+0000 7f94e216e700 1 --2- 172.21.15.17:0/89892105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc06a7e0 0x7f94dc10f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.750+0000 7f94e216e700 1 --2- 172.21.15.17:0/89892105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94dc1081e0 0x7f94dc1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.750+0000 7f94e216e700 1 --2- 172.21.15.17:0/89892105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94dc069e40 0x7f94dc06a2a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.750+0000 7f94e216e700 1 -- 172.21.15.17:0/89892105 >> 172.21.15.17:0/89892105 conn(0x7f94dc077fc0 msgr2=0x7f94dc0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.750+0000 7f94e216e700 1 -- 172.21.15.17:0/89892105 shutdown_connections 2024-03-24T03:00:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.750+0000 7f94e216e700 1 -- 172.21.15.17:0/89892105 wait complete. 2024-03-24T03:00:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94e216e700 1 Processor -- start 2024-03-24T03:00:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94e216e700 1 -- start start 2024-03-24T03:00:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94e216e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc069e40 0x7f94dc1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94e216e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94dc06a7e0 0x7f94dc1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94e216e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94dc1081e0 0x7f94dc19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94e216e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc112440 con 0x7f94dc1081e0 2024-03-24T03:00:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94e216e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc1122c0 con 0x7f94dc069e40 2024-03-24T03:00:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94e216e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc1125c0 con 0x7f94dc06a7e0 2024-03-24T03:00:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94dbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94dc1081e0 0x7f94dc19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94dbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94dc1081e0 0x7f94dc19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53014/0 (socket says 172.21.15.17:53014) 2024-03-24T03:00:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94dbfff700 1 -- 172.21.15.17:0/297309596 learned_addr learned my addr 172.21.15.17:0/297309596 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94daffd700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94dc06a7e0 0x7f94dc1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.751+0000 7f94db7fe700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc069e40 0x7f94dc1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:11.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94dbfff700 1 -- 172.21.15.17:0/297309596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94dc06a7e0 msgr2=0x7f94dc1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:11.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94dbfff700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94dc06a7e0 0x7f94dc1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:11.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94dbfff700 1 -- 172.21.15.17:0/297309596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc069e40 msgr2=0x7f94dc1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:11.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94dbfff700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc069e40 0x7f94dc1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:11.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94dbfff700 1 -- 172.21.15.17:0/297309596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc19b090 con 0x7f94dc1081e0 2024-03-24T03:00:11.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94daffd700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94dc06a7e0 0x7f94dc1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:11.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94db7fe700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc069e40 0x7f94dc1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:11.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94dbfff700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94dc1081e0 0x7f94dc19a930 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00e940 tx=0x7f94cc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94d8ff9700 1 -- 172.21.15.17:0/297309596 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc004a30 con 0x7f94dc1081e0 2024-03-24T03:00:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94e216e700 1 -- 172.21.15.17:0/297309596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94dc19b380 con 0x7f94dc1081e0 2024-03-24T03:00:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94d8ff9700 1 -- 172.21.15.17:0/297309596 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94cc005500 con 0x7f94dc1081e0 2024-03-24T03:00:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94d8ff9700 1 -- 172.21.15.17:0/297309596 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc00ce20 con 0x7f94dc1081e0 2024-03-24T03:00:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.752+0000 7f94e216e700 1 -- 172.21.15.17:0/297309596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94dc1a7640 con 0x7f94dc1081e0 2024-03-24T03:00:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.753+0000 7f94e216e700 1 -- 172.21.15.17:0/297309596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94dc068e50 con 0x7f94dc1081e0 2024-03-24T03:00:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.754+0000 7f94d8ff9700 1 -- 172.21.15.17:0/297309596 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f94cc005000 con 0x7f94dc1081e0 2024-03-24T03:00:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.755+0000 7f94d8ff9700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f94c80a2890 0x7f94c80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.755+0000 7f94db7fe700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f94c80a2890 0x7f94c80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.755+0000 7f94d8ff9700 1 -- 172.21.15.17:0/297309596 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f94cc0cf140 con 0x7f94dc1081e0 2024-03-24T03:00:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.755+0000 7f94db7fe700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f94c80a2890 0x7f94c80a4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f94dc19ba70 tx=0x7f94d400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:11.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.759+0000 7f94d8ff9700 1 -- 172.21.15.17:0/297309596 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f94cc053740 con 0x7f94dc1081e0 2024-03-24T03:00:11.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.958+0000 7f94e216e700 1 -- 172.21.15.17:0/297309596 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94dc10da80 con 0x7f94c80a2890 2024-03-24T03:00:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.968+0000 7f94d8ff9700 1 -- 172.21.15.17:0/297309596 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340515 (secure 0 0 0) 0x7f94dc10da80 con 0x7f94c80a2890 2024-03-24T03:00:11.972 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:11.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.971+0000 7f94e216e700 1 -- 172.21.15.17:0/297309596 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f94c80a2890 msgr2=0x7f94c80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.971+0000 7f94e216e700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f94c80a2890 0x7f94c80a4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f94dc19ba70 tx=0x7f94d400b040 comp rx=0 tx=0).stop 2024-03-24T03:00:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.971+0000 7f94e216e700 1 -- 172.21.15.17:0/297309596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94dc1081e0 msgr2=0x7f94dc19a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.971+0000 7f94e216e700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94dc1081e0 0x7f94dc19a930 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00e940 tx=0x7f94cc00ec50 comp rx=0 tx=0).stop 2024-03-24T03:00:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.971+0000 7f94e216e700 1 -- 172.21.15.17:0/297309596 shutdown_connections 2024-03-24T03:00:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.971+0000 7f94e216e700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f94c80a2890 0x7f94c80a4d50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.971+0000 7f94e216e700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94dc1081e0 0x7f94dc19a930 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.971+0000 7f94e216e700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc069e40 0x7f94dc1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.971+0000 7f94e216e700 1 --2- 172.21.15.17:0/297309596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94dc06a7e0 0x7f94dc1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.971+0000 7f94e216e700 1 -- 172.21.15.17:0/297309596 >> 172.21.15.17:0/297309596 conn(0x7f94dc077fc0 msgr2=0x7f94dc10d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.972+0000 7f94e216e700 1 -- 172.21.15.17:0/297309596 shutdown_connections 2024-03-24T03:00:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:11.972+0000 7f94e216e700 1 -- 172.21.15.17:0/297309596 wait complete. 2024-03-24T03:00:11.975 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-24T03:00:11.982 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:11 smithi017 ceph-mon[31926]: pgmap v341: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-24T03:00:12.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:11 smithi188 ceph-mon[36206]: pgmap v341: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-24T03:00:12.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:11 smithi019 ceph-mon[35944]: pgmap v341: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-24T03:00:12.715 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-03-24T03:00:09.971103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665081,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":873,"num_read_kb":565,"num_write":432,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":413324,"kb_used_data":14252,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124528500,"statfs":{"total":1151940427776,"available":1151517184000,"internally_reserved":0,"allocated":14594048,"data_stored":7979345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":408532281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":58,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003846"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.999713+0000","last_change":"2024-03-24T02:58:36.978486+0000","last_active":"2024-03-24T02:59:59.999713+0000","last_peered":"2024-03-24T02:59:59.999713+0000","last_clean":"2024-03-24T02:59:59.999713+0000","last_became_active":"2024-03-24T02:58:36.976937+0000","last_became_peered":"2024-03-24T02:58:36.976937+0000","last_unstale":"2024-03-24T02:59:59.999713+0000","last_undegraded":"2024-03-24T02:59:59.999713+0000","last_fullsized":"2024-03-24T02:59:59.999713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:25:09.439179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.994756+0000","last_change":"2024-03-24T02:58:36.983050+0000","last_active":"2024-03-24T02:59:59.994756+0000","last_peered":"2024-03-24T02:59:59.994756+0000","last_clean":"2024-03-24T02:59:59.994756+0000","last_became_active":"2024-03-24T02:58:36.982741+0000","last_became_peered":"2024-03-24T02:58:36.982741+0000","last_unstale":"2024-03-24T02:59:59.994756+0000","last_undegraded":"2024-03-24T02:59:59.994756+0000","last_fullsized":"2024-03-24T02:59:59.994756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:26:35.938559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.999873+0000","last_change":"2024-03-24T02:58:36.976313+0000","last_active":"2024-03-24T02:59:59.999873+0000","last_peered":"2024-03-24T02:59:59.999873+0000","last_clean":"2024-03-24T02:59:59.999873+0000","last_became_active":"2024-03-24T02:58:36.975087+0000","last_became_peered":"2024-03-24T02:58:36.975087+0000","last_unstale":"2024-03-24T02:59:59.999873+0000","last_undegraded":"2024-03-24T02:59:59.999873+0000","last_fullsized":"2024-03-24T02:59:59.999873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:22:31.476824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995743+0000","last_change":"2024-03-24T02:58:36.977632+0000","last_active":"2024-03-24T02:59:59.995743+0000","last_peered":"2024-03-24T02:59:59.995743+0000","last_clean":"2024-03-24T02:59:59.995743+0000","last_became_active":"2024-03-24T02:58:36.977111+0000","last_became_peered":"2024-03-24T02:58:36.977111+0000","last_unstale":"2024-03-24T02:59:59.995743+0000","last_undegraded":"2024-03-24T02:59:59.995743+0000","last_fullsized":"2024-03-24T02:59:59.995743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:47:07.874538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.990749+0000","last_change":"2024-03-24T02:58:36.967543+0000","last_active":"2024-03-24T02:59:59.990749+0000","last_peered":"2024-03-24T02:59:59.990749+0000","last_clean":"2024-03-24T02:59:59.990749+0000","last_became_active":"2024-03-24T02:58:36.967108+0000","last_became_peered":"2024-03-24T02:58:36.967108+0000","last_unstale":"2024-03-24T02:59:59.990749+0000","last_undegraded":"2024-03-24T02:59:59.990749+0000","last_fullsized":"2024-03-24T02:59:59.990749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:38:46.280891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.980432+0000","last_change":"2024-03-24T02:58:36.977777+0000","last_active":"2024-03-24T02:59:59.980432+0000","last_peered":"2024-03-24T02:59:59.980432+0000","last_clean":"2024-03-24T02:59:59.980432+0000","last_became_active":"2024-03-24T02:58:36.976615+0000","last_became_peered":"2024-03-24T02:58:36.976615+0000","last_unstale":"2024-03-24T02:59:59.980432+0000","last_undegraded":"2024-03-24T02:59:59.980432+0000","last_fullsized":"2024-03-24T02:59:59.980432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:13:15.149963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995932+0000","last_change":"2024-03-24T02:58:36.990832+0000","last_active":"2024-03-24T02:59:59.995932+0000","last_peered":"2024-03-24T02:59:59.995932+0000","last_clean":"2024-03-24T02:59:59.995932+0000","last_became_active":"2024-03-24T02:58:36.990395+0000","last_became_peered":"2024-03-24T02:58:36.990395+0000","last_unstale":"2024-03-24T02:59:59.995932+0000","last_undegraded":"2024-03-24T02:59:59.995932+0000","last_fullsized":"2024-03-24T02:59:59.995932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:34.915192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.982699+0000","last_change":"2024-03-24T02:59:31.132828+0000","last_active":"2024-03-24T02:59:59.982699+0000","last_peered":"2024-03-24T02:59:59.982699+0000","last_clean":"2024-03-24T02:59:59.982699+0000","last_became_active":"2024-03-24T02:59:31.132439+0000","last_became_peered":"2024-03-24T02:59:31.132439+0000","last_unstale":"2024-03-24T02:59:59.982699+0000","last_undegraded":"2024-03-24T02:59:59.982699+0000","last_fullsized":"2024-03-24T02:59:59.982699+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:06:57.643149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,10],"acting":[9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:00.000631+0000","last_change":"2024-03-24T02:58:36.989173+0000","last_active":"2024-03-24T03:00:00.000631+0000","last_peered":"2024-03-24T03:00:00.000631+0000","last_clean":"2024-03-24T03:00:00.000631+0000","last_became_active":"2024-03-24T02:58:36.988771+0000","last_became_peered":"2024-03-24T02:58:36.988771+0000","last_unstale":"2024-03-24T03:00:00.000631+0000","last_undegraded":"2024-03-24T03:00:00.000631+0000","last_fullsized":"2024-03-24T03:00:00.000631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:51:01.790952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.989886+0000","last_change":"2024-03-24T02:58:36.978505+0000","last_active":"2024-03-24T02:59:59.989886+0000","last_peered":"2024-03-24T02:59:59.989886+0000","last_clean":"2024-03-24T02:59:59.989886+0000","last_became_active":"2024-03-24T02:58:36.978145+0000","last_became_peered":"2024-03-24T02:58:36.978145+0000","last_unstale":"2024-03-24T02:59:59.989886+0000","last_undegraded":"2024-03-24T02:59:59.989886+0000","last_fullsized":"2024-03-24T02:59:59.989886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:06:41.874675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.975357+0000","last_change":"2024-03-24T02:59:31.616102+0000","last_active":"2024-03-24T02:59:59.975357+0000","last_peered":"2024-03-24T02:59:59.975357+0000","last_clean":"2024-03-24T02:59:59.975357+0000","last_became_active":"2024-03-24T02:59:31.615741+0000","last_became_peered":"2024-03-24T02:59:31.615741+0000","last_unstale":"2024-03-24T02:59:59.975357+0000","last_undegraded":"2024-03-24T02:59:59.975357+0000","last_fullsized":"2024-03-24T02:59:59.975357+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:04:55.547648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,9],"acting":[7,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.985399+0000","last_change":"2024-03-24T02:58:36.979852+0000","last_active":"2024-03-24T02:59:59.985399+0000","last_peered":"2024-03-24T02:59:59.985399+0000","last_clean":"2024-03-24T02:59:59.985399+0000","last_became_active":"2024-03-24T02:58:36.979506+0000","last_became_peered":"2024-03-24T02:58:36.979506+0000","last_unstale":"2024-03-24T02:59:59.985399+0000","last_undegraded":"2024-03-24T02:59:59.985399+0000","last_fullsized":"2024-03-24T02:59:59.985399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:01:27.493694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.996654+0000","last_change":"2024-03-24T02:58:36.977018+0000","last_active":"2024-03-24T02:59:59.996654+0000","last_peered":"2024-03-24T02:59:59.996654+0000","last_clean":"2024-03-24T02:59:59.996654+0000","last_became_active":"2024-03-24T02:58:36.976669+0000","last_became_peered":"2024-03-24T02:58:36.976669+0000","last_unstale":"2024-03-24T02:59:59.996654+0000","last_undegraded":"2024-03-24T02:59:59.996654+0000","last_fullsized":"2024-03-24T02:59:59.996654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:23:15.049093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995265+0000","last_change":"2024-03-24T02:58:36.973200+0000","last_active":"2024-03-24T02:59:59.995265+0000","last_peered":"2024-03-24T02:59:59.995265+0000","last_clean":"2024-03-24T02:59:59.995265+0000","last_became_active":"2024-03-24T02:58:36.972828+0000","last_became_peered":"2024-03-24T02:58:36.972828+0000","last_unstale":"2024-03-24T02:59:59.995265+0000","last_undegraded":"2024-03-24T02:59:59.995265+0000","last_fullsized":"2024-03-24T02:59:59.995265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:42:25.122881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.994823+0000","last_change":"2024-03-24T02:58:36.977686+0000","last_active":"2024-03-24T02:59:59.994823+0000","last_peered":"2024-03-24T02:59:59.994823+0000","last_clean":"2024-03-24T02:59:59.994823+0000","last_became_active":"2024-03-24T02:58:36.976311+0000","last_became_peered":"2024-03-24T02:58:36.976311+0000","last_unstale":"2024-03-24T02:59:59.994823+0000","last_undegraded":"2024-03-24T02:59:59.994823+0000","last_fullsized":"2024-03-24T02:59:59.994823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:10:04.585726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.999935+0000","last_change":"2024-03-24T02:58:36.985336+0000","last_active":"2024-03-24T02:59:59.999935+0000","last_peered":"2024-03-24T02:59:59.999935+0000","last_clean":"2024-03-24T02:59:59.999935+0000","last_became_active":"2024-03-24T02:58:36.984842+0000","last_became_peered":"2024-03-24T02:58:36.984842+0000","last_unstale":"2024-03-24T02:59:59.999935+0000","last_undegraded":"2024-03-24T02:59:59.999935+0000","last_fullsized":"2024-03-24T02:59:59.999935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:52:41.819132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:00.000107+0000","last_change":"2024-03-24T02:58:36.978310+0000","last_active":"2024-03-24T03:00:00.000107+0000","last_peered":"2024-03-24T03:00:00.000107+0000","last_clean":"2024-03-24T03:00:00.000107+0000","last_became_active":"2024-03-24T02:58:36.977396+0000","last_became_peered":"2024-03-24T02:58:36.977396+0000","last_unstale":"2024-03-24T03:00:00.000107+0000","last_undegraded":"2024-03-24T03:00:00.000107+0000","last_fullsized":"2024-03-24T03:00:00.000107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:37:21.200591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.972767+0000","last_change":"2024-03-24T02:58:36.979514+0000","last_active":"2024-03-24T02:59:59.972767+0000","last_peered":"2024-03-24T02:59:59.972767+0000","last_clean":"2024-03-24T02:59:59.972767+0000","last_became_active":"2024-03-24T02:58:36.979155+0000","last_became_peered":"2024-03-24T02:58:36.979155+0000","last_unstale":"2024-03-24T02:59:59.972767+0000","last_undegraded":"2024-03-24T02:59:59.972767+0000","last_fullsized":"2024-03-24T02:59:59.972767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:49:37.142195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.984643+0000","last_change":"2024-03-24T02:58:36.971546+0000","last_active":"2024-03-24T02:59:59.984643+0000","last_peered":"2024-03-24T02:59:59.984643+0000","last_clean":"2024-03-24T02:59:59.984643+0000","last_became_active":"2024-03-24T02:58:36.971210+0000","last_became_peered":"2024-03-24T02:58:36.971210+0000","last_unstale":"2024-03-24T02:59:59.984643+0000","last_undegraded":"2024-03-24T02:59:59.984643+0000","last_fullsized":"2024-03-24T02:59:59.984643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:35:21.274898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.969927+0000","last_change":"2024-03-24T02:58:40.994331+0000","last_active":"2024-03-24T02:59:30.969927+0000","last_peered":"2024-03-24T02:59:30.969927+0000","last_clean":"2024-03-24T02:59:30.969927+0000","last_became_active":"2024-03-24T02:58:40.993880+0000","last_became_peered":"2024-03-24T02:58:40.993880+0000","last_unstale":"2024-03-24T02:59:30.969927+0000","last_undegraded":"2024-03-24T02:59:30.969927+0000","last_fullsized":"2024-03-24T02:59:30.969927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:58:46.876872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:05.166402+0000","last_change":"2024-03-24T02:58:36.132975+0000","last_active":"2024-03-24T03:00:05.166402+0000","last_peered":"2024-03-24T03:00:05.166402+0000","last_clean":"2024-03-24T03:00:05.166402+0000","last_became_active":"2024-03-24T02:58:36.132640+0000","last_became_peered":"2024-03-24T02:58:36.132640+0000","last_unstale":"2024-03-24T03:00:05.166402+0000","last_undegraded":"2024-03-24T03:00:05.166402+0000","last_fullsized":"2024-03-24T03:00:05.166402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:37:03.765773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.983286+0000","last_change":"2024-03-24T02:59:30.973326+0000","last_active":"2024-03-24T02:59:59.983286+0000","last_peered":"2024-03-24T02:59:59.983286+0000","last_clean":"2024-03-24T02:59:59.983286+0000","last_became_active":"2024-03-24T02:59:30.972939+0000","last_became_peered":"2024-03-24T02:59:30.972939+0000","last_unstale":"2024-03-24T02:59:59.983286+0000","last_undegraded":"2024-03-24T02:59:59.983286+0000","last_fullsized":"2024-03-24T02:59:59.983286+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:53:31.582648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970362+0000","last_change":"2024-03-24T02:58:40.997896+0000","last_active":"2024-03-24T02:59:30.970362+0000","last_peered":"2024-03-24T02:59:30.970362+0000","last_clean":"2024-03-24T02:59:30.970362+0000","last_became_active":"2024-03-24T02:58:40.996845+0000","last_became_peered":"2024-03-24T02:58:40.996845+0000","last_unstale":"2024-03-24T02:59:30.970362+0000","last_undegraded":"2024-03-24T02:59:30.970362+0000","last_fullsized":"2024-03-24T02:59:30.970362+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:51:17.031332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.973187+0000","last_change":"2024-03-24T02:58:36.139069+0000","last_active":"2024-03-24T02:59:30.973187+0000","last_peered":"2024-03-24T02:59:30.973187+0000","last_clean":"2024-03-24T02:59:30.973187+0000","last_became_active":"2024-03-24T02:58:36.138571+0000","last_became_peered":"2024-03-24T02:58:36.138571+0000","last_unstale":"2024-03-24T02:59:30.973187+0000","last_undegraded":"2024-03-24T02:59:30.973187+0000","last_fullsized":"2024-03-24T02:59:30.973187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:17:44.510904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.999702+0000","last_change":"2024-03-24T02:58:36.976727+0000","last_active":"2024-03-24T02:59:59.999702+0000","last_peered":"2024-03-24T02:59:59.999702+0000","last_clean":"2024-03-24T02:59:59.999702+0000","last_became_active":"2024-03-24T02:58:36.976076+0000","last_became_peered":"2024-03-24T02:58:36.976076+0000","last_unstale":"2024-03-24T02:59:59.999702+0000","last_undegraded":"2024-03-24T02:59:59.999702+0000","last_fullsized":"2024-03-24T02:59:59.999702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:33:03.490882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970899+0000","last_change":"2024-03-24T02:58:41.003025+0000","last_active":"2024-03-24T02:59:30.970899+0000","last_peered":"2024-03-24T02:59:30.970899+0000","last_clean":"2024-03-24T02:59:30.970899+0000","last_became_active":"2024-03-24T02:58:41.002270+0000","last_became_peered":"2024-03-24T02:58:41.002270+0000","last_unstale":"2024-03-24T02:59:30.970899+0000","last_undegraded":"2024-03-24T02:59:30.970899+0000","last_fullsized":"2024-03-24T02:59:30.970899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:31:36.390191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.615123+0000","last_change":"2024-03-24T02:58:36.135836+0000","last_active":"2024-03-24T02:59:31.615123+0000","last_peered":"2024-03-24T02:59:31.615123+0000","last_clean":"2024-03-24T02:59:31.615123+0000","last_became_active":"2024-03-24T02:58:36.135476+0000","last_became_peered":"2024-03-24T02:58:36.135476+0000","last_unstale":"2024-03-24T02:59:31.615123+0000","last_undegraded":"2024-03-24T02:59:31.615123+0000","last_fullsized":"2024-03-24T02:59:31.615123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:30:14.058373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:00.002118+0000","last_change":"2024-03-24T02:58:36.981600+0000","last_active":"2024-03-24T03:00:00.002118+0000","last_peered":"2024-03-24T03:00:00.002118+0000","last_clean":"2024-03-24T03:00:00.002118+0000","last_became_active":"2024-03-24T02:58:36.981222+0000","last_became_peered":"2024-03-24T02:58:36.981222+0000","last_unstale":"2024-03-24T03:00:00.002118+0000","last_undegraded":"2024-03-24T03:00:00.002118+0000","last_fullsized":"2024-03-24T03:00:00.002118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:11:33.771157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970402+0000","last_change":"2024-03-24T02:58:40.996838+0000","last_active":"2024-03-24T02:59:30.970402+0000","last_peered":"2024-03-24T02:59:30.970402+0000","last_clean":"2024-03-24T02:59:30.970402+0000","last_became_active":"2024-03-24T02:58:40.996320+0000","last_became_peered":"2024-03-24T02:58:40.996320+0000","last_unstale":"2024-03-24T02:59:30.970402+0000","last_undegraded":"2024-03-24T02:59:30.970402+0000","last_fullsized":"2024-03-24T02:59:30.970402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:11:34.124402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.614884+0000","last_change":"2024-03-24T02:58:36.132406+0000","last_active":"2024-03-24T02:59:31.614884+0000","last_peered":"2024-03-24T02:59:31.614884+0000","last_clean":"2024-03-24T02:59:31.614884+0000","last_became_active":"2024-03-24T02:58:36.132044+0000","last_became_peered":"2024-03-24T02:58:36.132044+0000","last_unstale":"2024-03-24T02:59:31.614884+0000","last_undegraded":"2024-03-24T02:59:31.614884+0000","last_fullsized":"2024-03-24T02:59:31.614884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:35:53.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.999956+0000","last_change":"2024-03-24T02:58:36.977989+0000","last_active":"2024-03-24T02:59:59.999956+0000","last_peered":"2024-03-24T02:59:59.999956+0000","last_clean":"2024-03-24T02:59:59.999956+0000","last_became_active":"2024-03-24T02:58:36.977653+0000","last_became_peered":"2024-03-24T02:58:36.977653+0000","last_unstale":"2024-03-24T02:59:59.999956+0000","last_undegraded":"2024-03-24T02:59:59.999956+0000","last_fullsized":"2024-03-24T02:59:59.999956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:21:18.364057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.969630+0000","last_change":"2024-03-24T02:58:40.989082+0000","last_active":"2024-03-24T02:59:30.969630+0000","last_peered":"2024-03-24T02:59:30.969630+0000","last_clean":"2024-03-24T02:59:30.969630+0000","last_became_active":"2024-03-24T02:58:40.988731+0000","last_became_peered":"2024-03-24T02:58:40.988731+0000","last_unstale":"2024-03-24T02:59:30.969630+0000","last_undegraded":"2024-03-24T02:59:30.969630+0000","last_fullsized":"2024-03-24T02:59:30.969630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:34:39.489446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:07.166821+0000","last_change":"2024-03-24T02:58:36.135785+0000","last_active":"2024-03-24T03:00:07.166821+0000","last_peered":"2024-03-24T03:00:07.166821+0000","last_clean":"2024-03-24T03:00:07.166821+0000","last_became_active":"2024-03-24T02:58:36.135470+0000","last_became_peered":"2024-03-24T02:58:36.135470+0000","last_unstale":"2024-03-24T03:00:07.166821+0000","last_undegraded":"2024-03-24T03:00:07.166821+0000","last_fullsized":"2024-03-24T03:00:07.166821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:41:26.529773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.975619+0000","last_change":"2024-03-24T02:58:36.978241+0000","last_active":"2024-03-24T02:59:59.975619+0000","last_peered":"2024-03-24T02:59:59.975619+0000","last_clean":"2024-03-24T02:59:59.975619+0000","last_became_active":"2024-03-24T02:58:36.977879+0000","last_became_peered":"2024-03-24T02:58:36.977879+0000","last_unstale":"2024-03-24T02:59:59.975619+0000","last_undegraded":"2024-03-24T02:59:59.975619+0000","last_fullsized":"2024-03-24T02:59:59.975619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:44:41.190736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.616395+0000","last_change":"2024-03-24T02:59:31.616395+0000","last_active":"2024-03-24T02:59:31.616395+0000","last_peered":"2024-03-24T02:59:31.616395+0000","last_clean":"2024-03-24T02:59:31.616395+0000","last_became_active":"2024-03-24T02:59:31.615925+0000","last_became_peered":"2024-03-24T02:59:31.615925+0000","last_unstale":"2024-03-24T02:59:31.616395+0000","last_undegraded":"2024-03-24T02:59:31.616395+0000","last_fullsized":"2024-03-24T02:59:31.616395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:54:31.269893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.127563+0000","last_change":"2024-03-24T02:58:36.132242+0000","last_active":"2024-03-24T02:59:31.127563+0000","last_peered":"2024-03-24T02:59:31.127563+0000","last_clean":"2024-03-24T02:59:31.127563+0000","last_became_active":"2024-03-24T02:58:36.131792+0000","last_became_peered":"2024-03-24T02:58:36.131792+0000","last_unstale":"2024-03-24T02:59:31.127563+0000","last_undegraded":"2024-03-24T02:59:31.127563+0000","last_fullsized":"2024-03-24T02:59:31.127563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:09:16.751023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.997565+0000","last_change":"2024-03-24T02:58:36.977306+0000","last_active":"2024-03-24T02:59:59.997565+0000","last_peered":"2024-03-24T02:59:59.997565+0000","last_clean":"2024-03-24T02:59:59.997565+0000","last_became_active":"2024-03-24T02:58:36.976955+0000","last_became_peered":"2024-03-24T02:58:36.976955+0000","last_unstale":"2024-03-24T02:59:59.997565+0000","last_undegraded":"2024-03-24T02:59:59.997565+0000","last_fullsized":"2024-03-24T02:59:59.997565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:09:56.568781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.362779+0000","last_change":"2024-03-24T02:58:40.999977+0000","last_active":"2024-03-24T02:59:31.362779+0000","last_peered":"2024-03-24T02:59:31.362779+0000","last_clean":"2024-03-24T02:59:31.362779+0000","last_became_active":"2024-03-24T02:58:40.999564+0000","last_became_peered":"2024-03-24T02:58:40.999564+0000","last_unstale":"2024-03-24T02:59:31.362779+0000","last_undegraded":"2024-03-24T02:59:31.362779+0000","last_fullsized":"2024-03-24T02:59:31.362779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:16:58.022961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.131256+0000","last_change":"2024-03-24T02:58:36.123622+0000","last_active":"2024-03-24T02:59:31.131256+0000","last_peered":"2024-03-24T02:59:31.131256+0000","last_clean":"2024-03-24T02:59:31.131256+0000","last_became_active":"2024-03-24T02:58:36.123221+0000","last_became_peered":"2024-03-24T02:58:36.123221+0000","last_unstale":"2024-03-24T02:59:31.131256+0000","last_undegraded":"2024-03-24T02:59:31.131256+0000","last_fullsized":"2024-03-24T02:59:31.131256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:05:26.781268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.986606+0000","last_change":"2024-03-24T02:58:36.983446+0000","last_active":"2024-03-24T02:59:59.986606+0000","last_peered":"2024-03-24T02:59:59.986606+0000","last_clean":"2024-03-24T02:59:59.986606+0000","last_became_active":"2024-03-24T02:58:36.981401+0000","last_became_peered":"2024-03-24T02:58:36.981401+0000","last_unstale":"2024-03-24T02:59:59.986606+0000","last_undegraded":"2024-03-24T02:59:59.986606+0000","last_fullsized":"2024-03-24T02:59:59.986606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:12:42.836678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.972459+0000","last_change":"2024-03-24T02:58:40.999348+0000","last_active":"2024-03-24T02:59:30.972459+0000","last_peered":"2024-03-24T02:59:30.972459+0000","last_clean":"2024-03-24T02:59:30.972459+0000","last_became_active":"2024-03-24T02:58:40.998932+0000","last_became_peered":"2024-03-24T02:58:40.998932+0000","last_unstale":"2024-03-24T02:59:30.972459+0000","last_undegraded":"2024-03-24T02:59:30.972459+0000","last_fullsized":"2024-03-24T02:59:30.972459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:24:29.294464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971830+0000","last_change":"2024-03-24T02:58:36.137640+0000","last_active":"2024-03-24T02:59:30.971830+0000","last_peered":"2024-03-24T02:59:30.971830+0000","last_clean":"2024-03-24T02:59:30.971830+0000","last_became_active":"2024-03-24T02:58:36.137237+0000","last_became_peered":"2024-03-24T02:58:36.137237+0000","last_unstale":"2024-03-24T02:59:30.971830+0000","last_undegraded":"2024-03-24T02:59:30.971830+0000","last_fullsized":"2024-03-24T02:59:30.971830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:52:52.246968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.982845+0000","last_change":"2024-03-24T02:58:36.971696+0000","last_active":"2024-03-24T02:59:59.982845+0000","last_peered":"2024-03-24T02:59:59.982845+0000","last_clean":"2024-03-24T02:59:59.982845+0000","last_became_active":"2024-03-24T02:58:36.971318+0000","last_became_peered":"2024-03-24T02:58:36.971318+0000","last_unstale":"2024-03-24T02:59:59.982845+0000","last_undegraded":"2024-03-24T02:59:59.982845+0000","last_fullsized":"2024-03-24T02:59:59.982845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:01:30.976284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971021+0000","last_change":"2024-03-24T02:58:40.995887+0000","last_active":"2024-03-24T02:59:30.971021+0000","last_peered":"2024-03-24T02:59:30.971021+0000","last_clean":"2024-03-24T02:59:30.971021+0000","last_became_active":"2024-03-24T02:58:40.995242+0000","last_became_peered":"2024-03-24T02:58:40.995242+0000","last_unstale":"2024-03-24T02:59:30.971021+0000","last_undegraded":"2024-03-24T02:59:30.971021+0000","last_fullsized":"2024-03-24T02:59:30.971021+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:33:40.376374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":170,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:07.166340+0000","last_change":"2024-03-24T02:58:36.128252+0000","last_active":"2024-03-24T03:00:07.166340+0000","last_peered":"2024-03-24T03:00:07.166340+0000","last_clean":"2024-03-24T03:00:07.166340+0000","last_became_active":"2024-03-24T02:58:36.127888+0000","last_became_peered":"2024-03-24T02:58:36.127888+0000","last_unstale":"2024-03-24T03:00:07.166340+0000","last_undegraded":"2024-03-24T03:00:07.166340+0000","last_fullsized":"2024-03-24T03:00:07.166340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:31:39.341123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:00.001689+0000","last_change":"2024-03-24T02:58:36.983836+0000","last_active":"2024-03-24T03:00:00.001689+0000","last_peered":"2024-03-24T03:00:00.001689+0000","last_clean":"2024-03-24T03:00:00.001689+0000","last_became_active":"2024-03-24T02:58:36.983481+0000","last_became_peered":"2024-03-24T02:58:36.983481+0000","last_unstale":"2024-03-24T03:00:00.001689+0000","last_undegraded":"2024-03-24T03:00:00.001689+0000","last_fullsized":"2024-03-24T03:00:00.001689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:01:01.707796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.967591+0000","last_change":"2024-03-24T02:58:41.000176+0000","last_active":"2024-03-24T02:59:30.967591+0000","last_peered":"2024-03-24T02:59:30.967591+0000","last_clean":"2024-03-24T02:59:30.967591+0000","last_became_active":"2024-03-24T02:58:40.999561+0000","last_became_peered":"2024-03-24T02:58:40.999561+0000","last_unstale":"2024-03-24T02:59:30.967591+0000","last_undegraded":"2024-03-24T02:59:30.967591+0000","last_fullsized":"2024-03-24T02:59:30.967591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:20:28.678497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.974070+0000","last_change":"2024-03-24T02:58:36.131588+0000","last_active":"2024-03-24T02:59:30.974070+0000","last_peered":"2024-03-24T02:59:30.974070+0000","last_clean":"2024-03-24T02:59:30.974070+0000","last_became_active":"2024-03-24T02:58:36.131251+0000","last_became_peered":"2024-03-24T02:58:36.131251+0000","last_unstale":"2024-03-24T02:59:30.974070+0000","last_undegraded":"2024-03-24T02:59:30.974070+0000","last_fullsized":"2024-03-24T02:59:30.974070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:03:22.978352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995038+0000","last_change":"2024-03-24T02:59:31.616123+0000","last_active":"2024-03-24T02:59:59.995038+0000","last_peered":"2024-03-24T02:59:59.995038+0000","last_clean":"2024-03-24T02:59:59.995038+0000","last_became_active":"2024-03-24T02:59:31.615702+0000","last_became_peered":"2024-03-24T02:59:31.615702+0000","last_unstale":"2024-03-24T02:59:59.995038+0000","last_undegraded":"2024-03-24T02:59:59.995038+0000","last_fullsized":"2024-03-24T02:59:59.995038+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:06:00.354916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,8],"acting":[0,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970734+0000","last_change":"2024-03-24T02:58:41.003427+0000","last_active":"2024-03-24T02:59:30.970734+0000","last_peered":"2024-03-24T02:59:30.970734+0000","last_clean":"2024-03-24T02:59:30.970734+0000","last_became_active":"2024-03-24T02:58:41.002249+0000","last_became_peered":"2024-03-24T02:58:41.002249+0000","last_unstale":"2024-03-24T02:59:30.970734+0000","last_undegraded":"2024-03-24T02:59:30.970734+0000","last_fullsized":"2024-03-24T02:59:30.970734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:39:36.587451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970924+0000","last_change":"2024-03-24T02:58:36.132547+0000","last_active":"2024-03-24T02:59:30.970924+0000","last_peered":"2024-03-24T02:59:30.970924+0000","last_clean":"2024-03-24T02:59:30.970924+0000","last_became_active":"2024-03-24T02:58:36.132144+0000","last_became_peered":"2024-03-24T02:58:36.132144+0000","last_unstale":"2024-03-24T02:59:30.970924+0000","last_undegraded":"2024-03-24T02:59:30.970924+0000","last_fullsized":"2024-03-24T02:59:30.970924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:07:40.078639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:00.001269+0000","last_change":"2024-03-24T02:58:36.980107+0000","last_active":"2024-03-24T03:00:00.001269+0000","last_peered":"2024-03-24T03:00:00.001269+0000","last_clean":"2024-03-24T03:00:00.001269+0000","last_became_active":"2024-03-24T02:58:36.978914+0000","last_became_peered":"2024-03-24T02:58:36.978914+0000","last_unstale":"2024-03-24T03:00:00.001269+0000","last_undegraded":"2024-03-24T03:00:00.001269+0000","last_fullsized":"2024-03-24T03:00:00.001269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:43:36.956326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.967597+0000","last_change":"2024-03-24T02:58:40.996582+0000","last_active":"2024-03-24T02:59:30.967597+0000","last_peered":"2024-03-24T02:59:30.967597+0000","last_clean":"2024-03-24T02:59:30.967597+0000","last_became_active":"2024-03-24T02:58:40.993673+0000","last_became_peered":"2024-03-24T02:58:40.993673+0000","last_unstale":"2024-03-24T02:59:30.967597+0000","last_undegraded":"2024-03-24T02:59:30.967597+0000","last_fullsized":"2024-03-24T02:59:30.967597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:07:34.506740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":211,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:07.166353+0000","last_change":"2024-03-24T02:58:36.130829+0000","last_active":"2024-03-24T03:00:07.166353+0000","last_peered":"2024-03-24T03:00:07.166353+0000","last_clean":"2024-03-24T03:00:07.166353+0000","last_became_active":"2024-03-24T02:58:36.130483+0000","last_became_peered":"2024-03-24T02:58:36.130483+0000","last_unstale":"2024-03-24T03:00:07.166353+0000","last_undegraded":"2024-03-24T03:00:07.166353+0000","last_fullsized":"2024-03-24T03:00:07.166353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:40:32.407363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66993,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":146,"num_read_kb":138,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.978898+0000","last_change":"2024-03-24T02:58:36.984011+0000","last_active":"2024-03-24T02:59:59.978898+0000","last_peered":"2024-03-24T02:59:59.978898+0000","last_clean":"2024-03-24T02:59:59.978898+0000","last_became_active":"2024-03-24T02:58:36.983623+0000","last_became_peered":"2024-03-24T02:58:36.983623+0000","last_unstale":"2024-03-24T02:59:59.978898+0000","last_undegraded":"2024-03-24T02:59:59.978898+0000","last_fullsized":"2024-03-24T02:59:59.978898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:54:35.745458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971627+0000","last_change":"2024-03-24T02:58:40.993361+0000","last_active":"2024-03-24T02:59:30.971627+0000","last_peered":"2024-03-24T02:59:30.971627+0000","last_clean":"2024-03-24T02:59:30.971627+0000","last_became_active":"2024-03-24T02:58:40.992983+0000","last_became_peered":"2024-03-24T02:58:40.992983+0000","last_unstale":"2024-03-24T02:59:30.971627+0000","last_undegraded":"2024-03-24T02:59:30.971627+0000","last_fullsized":"2024-03-24T02:59:30.971627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:04:21.700904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.969985+0000","last_change":"2024-03-24T02:58:36.131101+0000","last_active":"2024-03-24T02:59:30.969985+0000","last_peered":"2024-03-24T02:59:30.969985+0000","last_clean":"2024-03-24T02:59:30.969985+0000","last_became_active":"2024-03-24T02:58:36.130356+0000","last_became_peered":"2024-03-24T02:58:36.130356+0000","last_unstale":"2024-03-24T02:59:30.969985+0000","last_undegraded":"2024-03-24T02:59:30.969985+0000","last_fullsized":"2024-03-24T02:59:30.969985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:16:40.613073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.992264+0000","last_change":"2024-03-24T02:58:36.983955+0000","last_active":"2024-03-24T02:59:59.992264+0000","last_peered":"2024-03-24T02:59:59.992264+0000","last_clean":"2024-03-24T02:59:59.992264+0000","last_became_active":"2024-03-24T02:58:36.983409+0000","last_became_peered":"2024-03-24T02:58:36.983409+0000","last_unstale":"2024-03-24T02:59:59.992264+0000","last_undegraded":"2024-03-24T02:59:59.992264+0000","last_fullsized":"2024-03-24T02:59:59.992264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:24:14.675779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.362637+0000","last_change":"2024-03-24T02:58:41.004609+0000","last_active":"2024-03-24T02:59:31.362637+0000","last_peered":"2024-03-24T02:59:31.362637+0000","last_clean":"2024-03-24T02:59:31.362637+0000","last_became_active":"2024-03-24T02:58:41.004199+0000","last_became_peered":"2024-03-24T02:58:41.004199+0000","last_unstale":"2024-03-24T02:59:31.362637+0000","last_undegraded":"2024-03-24T02:59:31.362637+0000","last_fullsized":"2024-03-24T02:59:31.362637+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:06:49.717152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971339+0000","last_change":"2024-03-24T02:58:36.137678+0000","last_active":"2024-03-24T02:59:30.971339+0000","last_peered":"2024-03-24T02:59:30.971339+0000","last_clean":"2024-03-24T02:59:30.971339+0000","last_became_active":"2024-03-24T02:58:36.137213+0000","last_became_peered":"2024-03-24T02:58:36.137213+0000","last_unstale":"2024-03-24T02:59:30.971339+0000","last_undegraded":"2024-03-24T02:59:30.971339+0000","last_fullsized":"2024-03-24T02:59:30.971339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:27:50.610128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.987518+0000","last_change":"2024-03-24T02:58:36.981976+0000","last_active":"2024-03-24T02:59:59.987518+0000","last_peered":"2024-03-24T02:59:59.987518+0000","last_clean":"2024-03-24T02:59:59.987518+0000","last_became_active":"2024-03-24T02:58:36.981617+0000","last_became_peered":"2024-03-24T02:58:36.981617+0000","last_unstale":"2024-03-24T02:59:59.987518+0000","last_undegraded":"2024-03-24T02:59:59.987518+0000","last_fullsized":"2024-03-24T02:59:59.987518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:39:43.822859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.968298+0000","last_change":"2024-03-24T02:58:41.001497+0000","last_active":"2024-03-24T02:59:30.968298+0000","last_peered":"2024-03-24T02:59:30.968298+0000","last_clean":"2024-03-24T02:59:30.968298+0000","last_became_active":"2024-03-24T02:58:41.001088+0000","last_became_peered":"2024-03-24T02:58:41.001088+0000","last_unstale":"2024-03-24T02:59:30.968298+0000","last_undegraded":"2024-03-24T02:59:30.968298+0000","last_fullsized":"2024-03-24T02:59:30.968298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:26:51.912623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":108,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:08.170282+0000","last_change":"2024-03-24T02:58:36.135245+0000","last_active":"2024-03-24T03:00:08.170282+0000","last_peered":"2024-03-24T03:00:08.170282+0000","last_clean":"2024-03-24T03:00:08.170282+0000","last_became_active":"2024-03-24T02:58:36.134853+0000","last_became_peered":"2024-03-24T02:58:36.134853+0000","last_unstale":"2024-03-24T03:00:08.170282+0000","last_undegraded":"2024-03-24T03:00:08.170282+0000","last_fullsized":"2024-03-24T03:00:08.170282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:50:11.890138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995857+0000","last_change":"2024-03-24T02:58:36.988925+0000","last_active":"2024-03-24T02:59:59.995857+0000","last_peered":"2024-03-24T02:59:59.995857+0000","last_clean":"2024-03-24T02:59:59.995857+0000","last_became_active":"2024-03-24T02:58:36.987201+0000","last_became_peered":"2024-03-24T02:58:36.987201+0000","last_unstale":"2024-03-24T02:59:59.995857+0000","last_undegraded":"2024-03-24T02:59:59.995857+0000","last_fullsized":"2024-03-24T02:59:59.995857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:38:54.277890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.131876+0000","last_change":"2024-03-24T02:58:41.002868+0000","last_active":"2024-03-24T02:59:31.131876+0000","last_peered":"2024-03-24T02:59:31.131876+0000","last_clean":"2024-03-24T02:59:31.131876+0000","last_became_active":"2024-03-24T02:58:41.002436+0000","last_became_peered":"2024-03-24T02:58:41.002436+0000","last_unstale":"2024-03-24T02:59:31.131876+0000","last_undegraded":"2024-03-24T02:59:31.131876+0000","last_fullsized":"2024-03-24T02:59:31.131876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:31:41.870769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.614592+0000","last_change":"2024-03-24T02:58:36.127705+0000","last_active":"2024-03-24T02:59:31.614592+0000","last_peered":"2024-03-24T02:59:31.614592+0000","last_clean":"2024-03-24T02:59:31.614592+0000","last_became_active":"2024-03-24T02:58:36.127359+0000","last_became_peered":"2024-03-24T02:58:36.127359+0000","last_unstale":"2024-03-24T02:59:31.614592+0000","last_undegraded":"2024-03-24T02:59:31.614592+0000","last_fullsized":"2024-03-24T02:59:31.614592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:14:13.504039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.997412+0000","last_change":"2024-03-24T02:58:36.986285+0000","last_active":"2024-03-24T02:59:59.997412+0000","last_peered":"2024-03-24T02:59:59.997412+0000","last_clean":"2024-03-24T02:59:59.997412+0000","last_became_active":"2024-03-24T02:58:36.983690+0000","last_became_peered":"2024-03-24T02:58:36.983690+0000","last_unstale":"2024-03-24T02:59:59.997412+0000","last_undegraded":"2024-03-24T02:59:59.997412+0000","last_fullsized":"2024-03-24T02:59:59.997412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:14:13.903312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970856+0000","last_change":"2024-03-24T02:58:40.994348+0000","last_active":"2024-03-24T02:59:30.970856+0000","last_peered":"2024-03-24T02:59:30.970856+0000","last_clean":"2024-03-24T02:59:30.970856+0000","last_became_active":"2024-03-24T02:58:40.993871+0000","last_became_peered":"2024-03-24T02:58:40.993871+0000","last_unstale":"2024-03-24T02:59:30.970856+0000","last_undegraded":"2024-03-24T02:59:30.970856+0000","last_fullsized":"2024-03-24T02:59:30.970856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:29:29.236204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.615185+0000","last_change":"2024-03-24T02:58:36.130645+0000","last_active":"2024-03-24T02:59:31.615185+0000","last_peered":"2024-03-24T02:59:31.615185+0000","last_clean":"2024-03-24T02:59:31.615185+0000","last_became_active":"2024-03-24T02:58:36.130289+0000","last_became_peered":"2024-03-24T02:58:36.130289+0000","last_unstale":"2024-03-24T02:59:31.615185+0000","last_undegraded":"2024-03-24T02:59:31.615185+0000","last_fullsized":"2024-03-24T02:59:31.615185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:18:04.085684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.984212+0000","last_change":"2024-03-24T02:58:36.980899+0000","last_active":"2024-03-24T02:59:59.984212+0000","last_peered":"2024-03-24T02:59:59.984212+0000","last_clean":"2024-03-24T02:59:59.984212+0000","last_became_active":"2024-03-24T02:58:36.979893+0000","last_became_peered":"2024-03-24T02:58:36.979893+0000","last_unstale":"2024-03-24T02:59:59.984212+0000","last_undegraded":"2024-03-24T02:59:59.984212+0000","last_fullsized":"2024-03-24T02:59:59.984212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:39.208269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.614857+0000","last_change":"2024-03-24T02:58:40.986381+0000","last_active":"2024-03-24T02:59:31.614857+0000","last_peered":"2024-03-24T02:59:31.614857+0000","last_clean":"2024-03-24T02:59:31.614857+0000","last_became_active":"2024-03-24T02:58:40.985539+0000","last_became_peered":"2024-03-24T02:58:40.985539+0000","last_unstale":"2024-03-24T02:59:31.614857+0000","last_undegraded":"2024-03-24T02:59:31.614857+0000","last_fullsized":"2024-03-24T02:59:31.614857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:10:13.593975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.967847+0000","last_change":"2024-03-24T02:58:36.126405+0000","last_active":"2024-03-24T02:59:30.967847+0000","last_peered":"2024-03-24T02:59:30.967847+0000","last_clean":"2024-03-24T02:59:30.967847+0000","last_became_active":"2024-03-24T02:58:36.126055+0000","last_became_peered":"2024-03-24T02:58:36.126055+0000","last_unstale":"2024-03-24T02:59:30.967847+0000","last_undegraded":"2024-03-24T02:59:30.967847+0000","last_fullsized":"2024-03-24T02:59:30.967847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:04:36.647320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.973993+0000","last_change":"2024-03-24T02:58:36.967546+0000","last_active":"2024-03-24T02:59:59.973993+0000","last_peered":"2024-03-24T02:59:59.973993+0000","last_clean":"2024-03-24T02:59:59.973993+0000","last_became_active":"2024-03-24T02:58:36.967158+0000","last_became_peered":"2024-03-24T02:58:36.967158+0000","last_unstale":"2024-03-24T02:59:59.973993+0000","last_undegraded":"2024-03-24T02:59:59.973993+0000","last_fullsized":"2024-03-24T02:59:59.973993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:06:43.500531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.131597+0000","last_change":"2024-03-24T02:58:41.005489+0000","last_active":"2024-03-24T02:59:31.131597+0000","last_peered":"2024-03-24T02:59:31.131597+0000","last_clean":"2024-03-24T02:59:31.131597+0000","last_became_active":"2024-03-24T02:58:41.004948+0000","last_became_peered":"2024-03-24T02:58:41.004948+0000","last_unstale":"2024-03-24T02:59:31.131597+0000","last_undegraded":"2024-03-24T02:59:31.131597+0000","last_fullsized":"2024-03-24T02:59:31.131597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:19:58.161486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:08.170222+0000","last_change":"2024-03-24T02:58:36.136598+0000","last_active":"2024-03-24T03:00:08.170222+0000","last_peered":"2024-03-24T03:00:08.170222+0000","last_clean":"2024-03-24T03:00:08.170222+0000","last_became_active":"2024-03-24T02:58:36.136210+0000","last_became_peered":"2024-03-24T02:58:36.136210+0000","last_unstale":"2024-03-24T03:00:08.170222+0000","last_undegraded":"2024-03-24T03:00:08.170222+0000","last_fullsized":"2024-03-24T03:00:08.170222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:28:51.216582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.979807+0000","last_change":"2024-03-24T02:58:36.976982+0000","last_active":"2024-03-24T02:59:59.979807+0000","last_peered":"2024-03-24T02:59:59.979807+0000","last_clean":"2024-03-24T02:59:59.979807+0000","last_became_active":"2024-03-24T02:58:36.976429+0000","last_became_peered":"2024-03-24T02:58:36.976429+0000","last_unstale":"2024-03-24T02:59:59.979807+0000","last_undegraded":"2024-03-24T02:59:59.979807+0000","last_fullsized":"2024-03-24T02:59:59.979807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:27:37.153099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.131944+0000","last_change":"2024-03-24T02:58:41.003368+0000","last_active":"2024-03-24T02:59:31.131944+0000","last_peered":"2024-03-24T02:59:31.131944+0000","last_clean":"2024-03-24T02:59:31.131944+0000","last_became_active":"2024-03-24T02:58:41.002950+0000","last_became_peered":"2024-03-24T02:58:41.002950+0000","last_unstale":"2024-03-24T02:59:31.131944+0000","last_undegraded":"2024-03-24T02:59:31.131944+0000","last_fullsized":"2024-03-24T02:59:31.131944+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:27:52.410491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.997471+0000","last_change":"2024-03-24T02:58:36.981969+0000","last_active":"2024-03-24T02:59:59.997471+0000","last_peered":"2024-03-24T02:59:59.997471+0000","last_clean":"2024-03-24T02:59:59.997471+0000","last_became_active":"2024-03-24T02:58:36.981535+0000","last_became_peered":"2024-03-24T02:58:36.981535+0000","last_unstale":"2024-03-24T02:59:59.997471+0000","last_undegraded":"2024-03-24T02:59:59.997471+0000","last_fullsized":"2024-03-24T02:59:59.997471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:45:02.572616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:07.168757+0000","last_change":"2024-03-24T02:58:36.130277+0000","last_active":"2024-03-24T03:00:07.168757+0000","last_peered":"2024-03-24T03:00:07.168757+0000","last_clean":"2024-03-24T03:00:07.168757+0000","last_became_active":"2024-03-24T02:58:36.129909+0000","last_became_peered":"2024-03-24T02:58:36.129909+0000","last_unstale":"2024-03-24T03:00:07.168757+0000","last_undegraded":"2024-03-24T03:00:07.168757+0000","last_fullsized":"2024-03-24T03:00:07.168757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:11:30.314752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.973996+0000","last_change":"2024-03-24T02:58:40.991037+0000","last_active":"2024-03-24T02:59:30.973996+0000","last_peered":"2024-03-24T02:59:30.973996+0000","last_clean":"2024-03-24T02:59:30.973996+0000","last_became_active":"2024-03-24T02:58:40.990382+0000","last_became_peered":"2024-03-24T02:58:40.990382+0000","last_unstale":"2024-03-24T02:59:30.973996+0000","last_undegraded":"2024-03-24T02:59:30.973996+0000","last_fullsized":"2024-03-24T02:59:30.973996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:08:00.048972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.996181+0000","last_change":"2024-03-24T02:58:36.976255+0000","last_active":"2024-03-24T02:59:59.996181+0000","last_peered":"2024-03-24T02:59:59.996181+0000","last_clean":"2024-03-24T02:59:59.996181+0000","last_became_active":"2024-03-24T02:58:36.975937+0000","last_became_peered":"2024-03-24T02:58:36.975937+0000","last_unstale":"2024-03-24T02:59:59.996181+0000","last_undegraded":"2024-03-24T02:59:59.996181+0000","last_fullsized":"2024-03-24T02:59:59.996181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:44:11.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970639+0000","last_change":"2024-03-24T02:58:36.126466+0000","last_active":"2024-03-24T02:59:30.970639+0000","last_peered":"2024-03-24T02:59:30.970639+0000","last_clean":"2024-03-24T02:59:30.970639+0000","last_became_active":"2024-03-24T02:58:36.125944+0000","last_became_peered":"2024-03-24T02:58:36.125944+0000","last_unstale":"2024-03-24T02:59:30.970639+0000","last_undegraded":"2024-03-24T02:59:30.970639+0000","last_fullsized":"2024-03-24T02:59:30.970639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:37:17.123640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.973356+0000","last_change":"2024-03-24T02:57:20.795020+0000","last_active":"2024-03-24T02:59:30.973356+0000","last_peered":"2024-03-24T02:59:30.973356+0000","last_clean":"2024-03-24T02:59:30.973356+0000","last_became_active":"2024-03-24T02:57:20.794535+0000","last_became_peered":"2024-03-24T02:57:20.794535+0000","last_unstale":"2024-03-24T02:59:30.973356+0000","last_undegraded":"2024-03-24T02:59:30.973356+0000","last_fullsized":"2024-03-24T02:59:30.973356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:53:54.827889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:53:54.827889+0000","last_clean_scrub_stamp":"2024-03-24T02:53:54.827889+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:47:16.501404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970983+0000","last_change":"2024-03-24T02:58:41.002766+0000","last_active":"2024-03-24T02:59:30.970983+0000","last_peered":"2024-03-24T02:59:30.970983+0000","last_clean":"2024-03-24T02:59:30.970983+0000","last_became_active":"2024-03-24T02:58:41.002401+0000","last_became_peered":"2024-03-24T02:58:41.002401+0000","last_unstale":"2024-03-24T02:59:30.970983+0000","last_undegraded":"2024-03-24T02:59:30.970983+0000","last_fullsized":"2024-03-24T02:59:30.970983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:58:31.401629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.990473+0000","last_change":"2024-03-24T02:58:36.978649+0000","last_active":"2024-03-24T02:59:59.990473+0000","last_peered":"2024-03-24T02:59:59.990473+0000","last_clean":"2024-03-24T02:59:59.990473+0000","last_became_active":"2024-03-24T02:58:36.978136+0000","last_became_peered":"2024-03-24T02:58:36.978136+0000","last_unstale":"2024-03-24T02:59:59.990473+0000","last_undegraded":"2024-03-24T02:59:59.990473+0000","last_fullsized":"2024-03-24T02:59:59.990473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:43:55.598250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.973361+0000","last_change":"2024-03-24T02:58:36.138886+0000","last_active":"2024-03-24T02:59:30.973361+0000","last_peered":"2024-03-24T02:59:30.973361+0000","last_clean":"2024-03-24T02:59:30.973361+0000","last_became_active":"2024-03-24T02:58:36.138582+0000","last_became_peered":"2024-03-24T02:58:36.138582+0000","last_unstale":"2024-03-24T02:59:30.973361+0000","last_undegraded":"2024-03-24T02:59:30.973361+0000","last_fullsized":"2024-03-24T02:59:30.973361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:17:48.129273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.362535+0000","last_change":"2024-03-24T02:58:41.000477+0000","last_active":"2024-03-24T02:59:31.362535+0000","last_peered":"2024-03-24T02:59:31.362535+0000","last_clean":"2024-03-24T02:59:31.362535+0000","last_became_active":"2024-03-24T02:58:41.000157+0000","last_became_peered":"2024-03-24T02:58:41.000157+0000","last_unstale":"2024-03-24T02:59:31.362535+0000","last_undegraded":"2024-03-24T02:59:31.362535+0000","last_fullsized":"2024-03-24T02:59:31.362535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:55:44.870347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.614937+0000","last_change":"2024-03-24T02:58:36.135524+0000","last_active":"2024-03-24T02:59:31.614937+0000","last_peered":"2024-03-24T02:59:31.614937+0000","last_clean":"2024-03-24T02:59:31.614937+0000","last_became_active":"2024-03-24T02:58:36.135170+0000","last_became_peered":"2024-03-24T02:58:36.135170+0000","last_unstale":"2024-03-24T02:59:31.614937+0000","last_undegraded":"2024-03-24T02:59:31.614937+0000","last_fullsized":"2024-03-24T02:59:31.614937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:56:52.737193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.990921+0000","last_change":"2024-03-24T02:58:36.969887+0000","last_active":"2024-03-24T02:59:59.990921+0000","last_peered":"2024-03-24T02:59:59.990921+0000","last_clean":"2024-03-24T02:59:59.990921+0000","last_became_active":"2024-03-24T02:58:36.969510+0000","last_became_peered":"2024-03-24T02:58:36.969510+0000","last_unstale":"2024-03-24T02:59:59.990921+0000","last_undegraded":"2024-03-24T02:59:59.990921+0000","last_fullsized":"2024-03-24T02:59:59.990921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:55:58.135452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970711+0000","last_change":"2024-03-24T02:58:40.996393+0000","last_active":"2024-03-24T02:59:30.970711+0000","last_peered":"2024-03-24T02:59:30.970711+0000","last_clean":"2024-03-24T02:59:30.970711+0000","last_became_active":"2024-03-24T02:58:40.995967+0000","last_became_peered":"2024-03-24T02:58:40.995967+0000","last_unstale":"2024-03-24T02:59:30.970711+0000","last_undegraded":"2024-03-24T02:59:30.970711+0000","last_fullsized":"2024-03-24T02:59:30.970711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:14:51.160730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.968113+0000","last_change":"2024-03-24T02:58:36.135033+0000","last_active":"2024-03-24T02:59:30.968113+0000","last_peered":"2024-03-24T02:59:30.968113+0000","last_clean":"2024-03-24T02:59:30.968113+0000","last_became_active":"2024-03-24T02:58:36.134713+0000","last_became_peered":"2024-03-24T02:58:36.134713+0000","last_unstale":"2024-03-24T02:59:30.968113+0000","last_undegraded":"2024-03-24T02:59:30.968113+0000","last_fullsized":"2024-03-24T02:59:30.968113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:05:55.777016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.993307+0000","last_change":"2024-03-24T02:58:36.984024+0000","last_active":"2024-03-24T02:59:59.993307+0000","last_peered":"2024-03-24T02:59:59.993307+0000","last_clean":"2024-03-24T02:59:59.993307+0000","last_became_active":"2024-03-24T02:58:36.983659+0000","last_became_peered":"2024-03-24T02:58:36.983659+0000","last_unstale":"2024-03-24T02:59:59.993307+0000","last_undegraded":"2024-03-24T02:59:59.993307+0000","last_fullsized":"2024-03-24T02:59:59.993307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:47:59.900749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.969725+0000","last_change":"2024-03-24T02:58:40.991181+0000","last_active":"2024-03-24T02:59:30.969725+0000","last_peered":"2024-03-24T02:59:30.969725+0000","last_clean":"2024-03-24T02:59:30.969725+0000","last_became_active":"2024-03-24T02:58:40.990755+0000","last_became_peered":"2024-03-24T02:58:40.990755+0000","last_unstale":"2024-03-24T02:59:30.969725+0000","last_undegraded":"2024-03-24T02:59:30.969725+0000","last_fullsized":"2024-03-24T02:59:30.969725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:57:05.480364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971003+0000","last_change":"2024-03-24T02:58:36.123100+0000","last_active":"2024-03-24T02:59:30.971003+0000","last_peered":"2024-03-24T02:59:30.971003+0000","last_clean":"2024-03-24T02:59:30.971003+0000","last_became_active":"2024-03-24T02:58:36.122613+0000","last_became_peered":"2024-03-24T02:58:36.122613+0000","last_unstale":"2024-03-24T02:59:30.971003+0000","last_undegraded":"2024-03-24T02:59:30.971003+0000","last_fullsized":"2024-03-24T02:59:30.971003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:31:23.954532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.997313+0000","last_change":"2024-03-24T02:58:36.977010+0000","last_active":"2024-03-24T02:59:59.997313+0000","last_peered":"2024-03-24T02:59:59.997313+0000","last_clean":"2024-03-24T02:59:59.997313+0000","last_became_active":"2024-03-24T02:58:36.976499+0000","last_became_peered":"2024-03-24T02:58:36.976499+0000","last_unstale":"2024-03-24T02:59:59.997313+0000","last_undegraded":"2024-03-24T02:59:59.997313+0000","last_fullsized":"2024-03-24T02:59:59.997313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:18:33.754474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971114+0000","last_change":"2024-03-24T02:58:40.991043+0000","last_active":"2024-03-24T02:59:30.971114+0000","last_peered":"2024-03-24T02:59:30.971114+0000","last_clean":"2024-03-24T02:59:30.971114+0000","last_became_active":"2024-03-24T02:58:40.990549+0000","last_became_peered":"2024-03-24T02:58:40.990549+0000","last_unstale":"2024-03-24T02:59:30.971114+0000","last_undegraded":"2024-03-24T02:59:30.971114+0000","last_fullsized":"2024-03-24T02:59:30.971114+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:20:34.014398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.436877+0000","last_change":"2024-03-24T02:58:36.127420+0000","last_active":"2024-03-24T02:59:31.436877+0000","last_peered":"2024-03-24T02:59:31.436877+0000","last_clean":"2024-03-24T02:59:31.436877+0000","last_became_active":"2024-03-24T02:58:36.126995+0000","last_became_peered":"2024-03-24T02:58:36.126995+0000","last_unstale":"2024-03-24T02:59:31.436877+0000","last_undegraded":"2024-03-24T02:59:31.436877+0000","last_fullsized":"2024-03-24T02:59:31.436877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:01:52.250569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.955504+0000","last_change":"2024-03-24T02:58:36.971978+0000","last_active":"2024-03-24T02:59:59.955504+0000","last_peered":"2024-03-24T02:59:59.955504+0000","last_clean":"2024-03-24T02:59:59.955504+0000","last_became_active":"2024-03-24T02:58:36.969812+0000","last_became_peered":"2024-03-24T02:58:36.969812+0000","last_unstale":"2024-03-24T02:59:59.955504+0000","last_undegraded":"2024-03-24T02:59:59.955504+0000","last_fullsized":"2024-03-24T02:59:59.955504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:44:38.573916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.968435+0000","last_change":"2024-03-24T02:58:40.994515+0000","last_active":"2024-03-24T02:59:30.968435+0000","last_peered":"2024-03-24T02:59:30.968435+0000","last_clean":"2024-03-24T02:59:30.968435+0000","last_became_active":"2024-03-24T02:58:40.994113+0000","last_became_peered":"2024-03-24T02:58:40.994113+0000","last_unstale":"2024-03-24T02:59:30.968435+0000","last_undegraded":"2024-03-24T02:59:30.968435+0000","last_fullsized":"2024-03-24T02:59:30.968435+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:21:58.972660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971131+0000","last_change":"2024-03-24T02:58:36.136590+0000","last_active":"2024-03-24T02:59:30.971131+0000","last_peered":"2024-03-24T02:59:30.971131+0000","last_clean":"2024-03-24T02:59:30.971131+0000","last_became_active":"2024-03-24T02:58:36.136248+0000","last_became_peered":"2024-03-24T02:58:36.136248+0000","last_unstale":"2024-03-24T02:59:30.971131+0000","last_undegraded":"2024-03-24T02:59:30.971131+0000","last_fullsized":"2024-03-24T02:59:30.971131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:20:50.738185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.994480+0000","last_change":"2024-03-24T02:58:36.977145+0000","last_active":"2024-03-24T02:59:59.994480+0000","last_peered":"2024-03-24T02:59:59.994480+0000","last_clean":"2024-03-24T02:59:59.994480+0000","last_became_active":"2024-03-24T02:58:36.975835+0000","last_became_peered":"2024-03-24T02:58:36.975835+0000","last_unstale":"2024-03-24T02:59:59.994480+0000","last_undegraded":"2024-03-24T02:59:59.994480+0000","last_fullsized":"2024-03-24T02:59:59.994480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:33:30.150675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970576+0000","last_change":"2024-03-24T02:58:41.005505+0000","last_active":"2024-03-24T02:59:30.970576+0000","last_peered":"2024-03-24T02:59:30.970576+0000","last_clean":"2024-03-24T02:59:30.970576+0000","last_became_active":"2024-03-24T02:58:41.005063+0000","last_became_peered":"2024-03-24T02:58:41.005063+0000","last_unstale":"2024-03-24T02:59:30.970576+0000","last_undegraded":"2024-03-24T02:59:30.970576+0000","last_fullsized":"2024-03-24T02:59:30.970576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:22:31.947209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.969661+0000","last_change":"2024-03-24T02:58:36.130936+0000","last_active":"2024-03-24T02:59:30.969661+0000","last_peered":"2024-03-24T02:59:30.969661+0000","last_clean":"2024-03-24T02:59:30.969661+0000","last_became_active":"2024-03-24T02:58:36.130581+0000","last_became_peered":"2024-03-24T02:58:36.130581+0000","last_unstale":"2024-03-24T02:59:30.969661+0000","last_undegraded":"2024-03-24T02:59:30.969661+0000","last_fullsized":"2024-03-24T02:59:30.969661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:52.958392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.981649+0000","last_change":"2024-03-24T02:58:36.977062+0000","last_active":"2024-03-24T02:59:59.981649+0000","last_peered":"2024-03-24T02:59:59.981649+0000","last_clean":"2024-03-24T02:59:59.981649+0000","last_became_active":"2024-03-24T02:58:36.976675+0000","last_became_peered":"2024-03-24T02:58:36.976675+0000","last_unstale":"2024-03-24T02:59:59.981649+0000","last_undegraded":"2024-03-24T02:59:59.981649+0000","last_fullsized":"2024-03-24T02:59:59.981649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:37:31.955379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970801+0000","last_change":"2024-03-24T02:58:40.996227+0000","last_active":"2024-03-24T02:59:30.970801+0000","last_peered":"2024-03-24T02:59:30.970801+0000","last_clean":"2024-03-24T02:59:30.970801+0000","last_became_active":"2024-03-24T02:58:40.995394+0000","last_became_peered":"2024-03-24T02:58:40.995394+0000","last_unstale":"2024-03-24T02:59:30.970801+0000","last_undegraded":"2024-03-24T02:59:30.970801+0000","last_fullsized":"2024-03-24T02:59:30.970801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:00:04.370712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.614783+0000","last_change":"2024-03-24T02:58:36.135777+0000","last_active":"2024-03-24T02:59:31.614783+0000","last_peered":"2024-03-24T02:59:31.614783+0000","last_clean":"2024-03-24T02:59:31.614783+0000","last_became_active":"2024-03-24T02:58:36.135398+0000","last_became_peered":"2024-03-24T02:58:36.135398+0000","last_unstale":"2024-03-24T02:59:31.614783+0000","last_undegraded":"2024-03-24T02:59:31.614783+0000","last_fullsized":"2024-03-24T02:59:31.614783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:11:01.641323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.985294+0000","last_change":"2024-03-24T02:58:36.981004+0000","last_active":"2024-03-24T02:59:59.985294+0000","last_peered":"2024-03-24T02:59:59.985294+0000","last_clean":"2024-03-24T02:59:59.985294+0000","last_became_active":"2024-03-24T02:58:36.980258+0000","last_became_peered":"2024-03-24T02:58:36.980258+0000","last_unstale":"2024-03-24T02:59:59.985294+0000","last_undegraded":"2024-03-24T02:59:59.985294+0000","last_fullsized":"2024-03-24T02:59:59.985294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:21:22.479122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.436837+0000","last_change":"2024-03-24T02:58:40.994636+0000","last_active":"2024-03-24T02:59:31.436837+0000","last_peered":"2024-03-24T02:59:31.436837+0000","last_clean":"2024-03-24T02:59:31.436837+0000","last_became_active":"2024-03-24T02:58:40.994122+0000","last_became_peered":"2024-03-24T02:58:40.994122+0000","last_unstale":"2024-03-24T02:59:31.436837+0000","last_undegraded":"2024-03-24T02:59:31.436837+0000","last_fullsized":"2024-03-24T02:59:31.436837+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:55:56.707430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":108,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:08.170254+0000","last_change":"2024-03-24T02:58:36.134528+0000","last_active":"2024-03-24T03:00:08.170254+0000","last_peered":"2024-03-24T03:00:08.170254+0000","last_clean":"2024-03-24T03:00:08.170254+0000","last_became_active":"2024-03-24T02:58:36.134158+0000","last_became_peered":"2024-03-24T02:58:36.134158+0000","last_unstale":"2024-03-24T03:00:08.170254+0000","last_undegraded":"2024-03-24T03:00:08.170254+0000","last_fullsized":"2024-03-24T03:00:08.170254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:50:31.582575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.985991+0000","last_change":"2024-03-24T02:58:36.984545+0000","last_active":"2024-03-24T02:59:59.985991+0000","last_peered":"2024-03-24T02:59:59.985991+0000","last_clean":"2024-03-24T02:59:59.985991+0000","last_became_active":"2024-03-24T02:58:36.982843+0000","last_became_peered":"2024-03-24T02:58:36.982843+0000","last_unstale":"2024-03-24T02:59:59.985991+0000","last_undegraded":"2024-03-24T02:59:59.985991+0000","last_fullsized":"2024-03-24T02:59:59.985991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:50:04.481097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970923+0000","last_change":"2024-03-24T02:58:41.000190+0000","last_active":"2024-03-24T02:59:30.970923+0000","last_peered":"2024-03-24T02:59:30.970923+0000","last_clean":"2024-03-24T02:59:30.970923+0000","last_became_active":"2024-03-24T02:58:40.999767+0000","last_became_peered":"2024-03-24T02:58:40.999767+0000","last_unstale":"2024-03-24T02:59:30.970923+0000","last_undegraded":"2024-03-24T02:59:30.970923+0000","last_fullsized":"2024-03-24T02:59:30.970923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:31:44.927277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971379+0000","last_change":"2024-03-24T02:58:36.133118+0000","last_active":"2024-03-24T02:59:30.971379+0000","last_peered":"2024-03-24T02:59:30.971379+0000","last_clean":"2024-03-24T02:59:30.971379+0000","last_became_active":"2024-03-24T02:58:36.132747+0000","last_became_peered":"2024-03-24T02:58:36.132747+0000","last_unstale":"2024-03-24T02:59:30.971379+0000","last_undegraded":"2024-03-24T02:59:30.971379+0000","last_fullsized":"2024-03-24T02:59:30.971379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:11:15.665601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.996234+0000","last_change":"2024-03-24T02:58:36.985489+0000","last_active":"2024-03-24T02:59:59.996234+0000","last_peered":"2024-03-24T02:59:59.996234+0000","last_clean":"2024-03-24T02:59:59.996234+0000","last_became_active":"2024-03-24T02:58:36.984421+0000","last_became_peered":"2024-03-24T02:58:36.984421+0000","last_unstale":"2024-03-24T02:59:59.996234+0000","last_undegraded":"2024-03-24T02:59:59.996234+0000","last_fullsized":"2024-03-24T02:59:59.996234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:57:03.004888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971389+0000","last_change":"2024-03-24T02:58:40.991456+0000","last_active":"2024-03-24T02:59:30.971389+0000","last_peered":"2024-03-24T02:59:30.971389+0000","last_clean":"2024-03-24T02:59:30.971389+0000","last_became_active":"2024-03-24T02:58:40.990973+0000","last_became_peered":"2024-03-24T02:58:40.990973+0000","last_unstale":"2024-03-24T02:59:30.971389+0000","last_undegraded":"2024-03-24T02:59:30.971389+0000","last_fullsized":"2024-03-24T02:59:30.971389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:36:01.276988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.131981+0000","last_change":"2024-03-24T02:58:36.130967+0000","last_active":"2024-03-24T02:59:31.131981+0000","last_peered":"2024-03-24T02:59:31.131981+0000","last_clean":"2024-03-24T02:59:31.131981+0000","last_became_active":"2024-03-24T02:58:36.130657+0000","last_became_peered":"2024-03-24T02:58:36.130657+0000","last_unstale":"2024-03-24T02:59:31.131981+0000","last_undegraded":"2024-03-24T02:59:31.131981+0000","last_fullsized":"2024-03-24T02:59:31.131981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:26:20.426922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.988460+0000","last_change":"2024-03-24T02:58:36.982297+0000","last_active":"2024-03-24T02:59:59.988460+0000","last_peered":"2024-03-24T02:59:59.988460+0000","last_clean":"2024-03-24T02:59:59.988460+0000","last_became_active":"2024-03-24T02:58:36.981792+0000","last_became_peered":"2024-03-24T02:58:36.981792+0000","last_unstale":"2024-03-24T02:59:59.988460+0000","last_undegraded":"2024-03-24T02:59:59.988460+0000","last_fullsized":"2024-03-24T02:59:59.988460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:03:07.528142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.996393+0000","last_change":"2024-03-24T02:58:36.979815+0000","last_active":"2024-03-24T02:59:59.996393+0000","last_peered":"2024-03-24T02:59:59.996393+0000","last_clean":"2024-03-24T02:59:59.996393+0000","last_became_active":"2024-03-24T02:58:36.979467+0000","last_became_peered":"2024-03-24T02:58:36.979467+0000","last_unstale":"2024-03-24T02:59:59.996393+0000","last_undegraded":"2024-03-24T02:59:59.996393+0000","last_fullsized":"2024-03-24T02:59:59.996393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:28.056630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.994790+0000","last_change":"2024-03-24T02:59:31.616266+0000","last_active":"2024-03-24T02:59:59.994790+0000","last_peered":"2024-03-24T02:59:59.994790+0000","last_clean":"2024-03-24T02:59:59.994790+0000","last_became_active":"2024-03-24T02:59:31.615872+0000","last_became_peered":"2024-03-24T02:59:31.615872+0000","last_unstale":"2024-03-24T02:59:59.994790+0000","last_undegraded":"2024-03-24T02:59:59.994790+0000","last_fullsized":"2024-03-24T02:59:59.994790+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:36:35.596949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.969229+0000","last_change":"2024-03-24T02:58:36.978207+0000","last_active":"2024-03-24T02:59:59.969229+0000","last_peered":"2024-03-24T02:59:59.969229+0000","last_clean":"2024-03-24T02:59:59.969229+0000","last_became_active":"2024-03-24T02:58:36.977760+0000","last_became_peered":"2024-03-24T02:58:36.977760+0000","last_unstale":"2024-03-24T02:59:59.969229+0000","last_undegraded":"2024-03-24T02:59:59.969229+0000","last_fullsized":"2024-03-24T02:59:59.969229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:43:51.603963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.994786+0000","last_change":"2024-03-24T02:59:31.132808+0000","last_active":"2024-03-24T02:59:59.994786+0000","last_peered":"2024-03-24T02:59:59.994786+0000","last_clean":"2024-03-24T02:59:59.994786+0000","last_became_active":"2024-03-24T02:59:31.132404+0000","last_became_peered":"2024-03-24T02:59:31.132404+0000","last_unstale":"2024-03-24T02:59:59.994786+0000","last_undegraded":"2024-03-24T02:59:59.994786+0000","last_fullsized":"2024-03-24T02:59:59.994786+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:55:07.143353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995763+0000","last_change":"2024-03-24T02:58:36.977261+0000","last_active":"2024-03-24T02:59:59.995763+0000","last_peered":"2024-03-24T02:59:59.995763+0000","last_clean":"2024-03-24T02:59:59.995763+0000","last_became_active":"2024-03-24T02:58:36.974644+0000","last_became_peered":"2024-03-24T02:58:36.974644+0000","last_unstale":"2024-03-24T02:59:59.995763+0000","last_undegraded":"2024-03-24T02:59:59.995763+0000","last_fullsized":"2024-03-24T02:59:59.995763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:34:50.968770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.984170+0000","last_change":"2024-03-24T02:58:36.981223+0000","last_active":"2024-03-24T02:59:59.984170+0000","last_peered":"2024-03-24T02:59:59.984170+0000","last_clean":"2024-03-24T02:59:59.984170+0000","last_became_active":"2024-03-24T02:58:36.979728+0000","last_became_peered":"2024-03-24T02:58:36.979728+0000","last_unstale":"2024-03-24T02:59:59.984170+0000","last_undegraded":"2024-03-24T02:59:59.984170+0000","last_fullsized":"2024-03-24T02:59:59.984170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:08:41.300442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.990102+0000","last_change":"2024-03-24T02:58:36.972385+0000","last_active":"2024-03-24T02:59:59.990102+0000","last_peered":"2024-03-24T02:59:59.990102+0000","last_clean":"2024-03-24T02:59:59.990102+0000","last_became_active":"2024-03-24T02:58:36.972018+0000","last_became_peered":"2024-03-24T02:58:36.972018+0000","last_unstale":"2024-03-24T02:59:59.990102+0000","last_undegraded":"2024-03-24T02:59:59.990102+0000","last_fullsized":"2024-03-24T02:59:59.990102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:02:22.529198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.987478+0000","last_change":"2024-03-24T02:59:31.132804+0000","last_active":"2024-03-24T02:59:59.987478+0000","last_peered":"2024-03-24T02:59:59.987478+0000","last_clean":"2024-03-24T02:59:59.987478+0000","last_became_active":"2024-03-24T02:59:31.132417+0000","last_became_peered":"2024-03-24T02:59:31.132417+0000","last_unstale":"2024-03-24T02:59:59.987478+0000","last_undegraded":"2024-03-24T02:59:59.987478+0000","last_fullsized":"2024-03-24T02:59:59.987478+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:17:03.492418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995489+0000","last_change":"2024-03-24T02:58:36.977256+0000","last_active":"2024-03-24T02:59:59.995489+0000","last_peered":"2024-03-24T02:59:59.995489+0000","last_clean":"2024-03-24T02:59:59.995489+0000","last_became_active":"2024-03-24T02:58:36.976888+0000","last_became_peered":"2024-03-24T02:58:36.976888+0000","last_unstale":"2024-03-24T02:59:59.995489+0000","last_undegraded":"2024-03-24T02:59:59.995489+0000","last_fullsized":"2024-03-24T02:59:59.995489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:18:19.449590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.987776+0000","last_change":"2024-03-24T02:58:36.981413+0000","last_active":"2024-03-24T02:59:59.987776+0000","last_peered":"2024-03-24T02:59:59.987776+0000","last_clean":"2024-03-24T02:59:59.987776+0000","last_became_active":"2024-03-24T02:58:36.980346+0000","last_became_peered":"2024-03-24T02:58:36.980346+0000","last_unstale":"2024-03-24T02:59:59.987776+0000","last_undegraded":"2024-03-24T02:59:59.987776+0000","last_fullsized":"2024-03-24T02:59:59.987776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:17:57.332228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:00.000008+0000","last_change":"2024-03-24T02:58:36.984264+0000","last_active":"2024-03-24T03:00:00.000008+0000","last_peered":"2024-03-24T03:00:00.000008+0000","last_clean":"2024-03-24T03:00:00.000008+0000","last_became_active":"2024-03-24T02:58:36.983905+0000","last_became_peered":"2024-03-24T02:58:36.983905+0000","last_unstale":"2024-03-24T03:00:00.000008+0000","last_undegraded":"2024-03-24T03:00:00.000008+0000","last_fullsized":"2024-03-24T03:00:00.000008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:25:17.872320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995968+0000","last_change":"2024-03-24T02:58:36.975939+0000","last_active":"2024-03-24T02:59:59.995968+0000","last_peered":"2024-03-24T02:59:59.995968+0000","last_clean":"2024-03-24T02:59:59.995968+0000","last_became_active":"2024-03-24T02:58:36.975477+0000","last_became_peered":"2024-03-24T02:58:36.975477+0000","last_unstale":"2024-03-24T02:59:59.995968+0000","last_undegraded":"2024-03-24T02:59:59.995968+0000","last_fullsized":"2024-03-24T02:59:59.995968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:46:43.603391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.991554+0000","last_change":"2024-03-24T02:58:36.980674+0000","last_active":"2024-03-24T02:59:59.991554+0000","last_peered":"2024-03-24T02:59:59.991554+0000","last_clean":"2024-03-24T02:59:59.991554+0000","last_became_active":"2024-03-24T02:58:36.980278+0000","last_became_peered":"2024-03-24T02:58:36.980278+0000","last_unstale":"2024-03-24T02:59:59.991554+0000","last_undegraded":"2024-03-24T02:59:59.991554+0000","last_fullsized":"2024-03-24T02:59:59.991554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:50:01.719362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74479,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":737,"num_read_kb":446,"num_write":249,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":223437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":501823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1155632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":505317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":501599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1098237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":496725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":564748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":498713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":565500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":496433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":501331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1658880,"data_stored":1093287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-24T03:00:12.721 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-24T03:00:12.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:12 smithi017 ceph-mon[31926]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T03:00:13.011 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:00:13.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:12 smithi188 ceph-mon[36206]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T03:00:13.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:12 smithi019 ceph-mon[35944]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T03:00:14.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:13 smithi188 ceph-mon[36206]: pgmap v342: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-24T03:00:14.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:13 smithi019 ceph-mon[35944]: pgmap v342: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-24T03:00:14.134 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:13 smithi017 ceph-mon[31926]: pgmap v342: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-24T03:00:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.724+0000 7fd3add7b700 1 -- 172.21.15.17:0/2078297320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3a8101070 msgr2=0x7fd3a81014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.724+0000 7fd3add7b700 1 --2- 172.21.15.17:0/2078297320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3a8101070 0x7fd3a81014f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009990 tx=0x7fd39c009ca0 comp rx=0 tx=0).stop 2024-03-24T03:00:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.724+0000 7fd3add7b700 1 -- 172.21.15.17:0/2078297320 shutdown_connections 2024-03-24T03:00:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.724+0000 7fd3add7b700 1 --2- 172.21.15.17:0/2078297320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3a8101a30 0x7fd3a810ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.724+0000 7fd3add7b700 1 --2- 172.21.15.17:0/2078297320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3a8107120 0x7fd3a8107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:14.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.724+0000 7fd3add7b700 1 --2- 172.21.15.17:0/2078297320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3a8101070 0x7fd3a81014f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:14.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.724+0000 7fd3add7b700 1 -- 172.21.15.17:0/2078297320 >> 172.21.15.17:0/2078297320 conn(0x7fd3a80780a0 msgr2=0x7fd3a80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:14.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.724+0000 7fd3add7b700 1 -- 172.21.15.17:0/2078297320 shutdown_connections 2024-03-24T03:00:14.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.724+0000 7fd3add7b700 1 -- 172.21.15.17:0/2078297320 wait complete. 2024-03-24T03:00:14.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.724+0000 7fd3add7b700 1 Processor -- start 2024-03-24T03:00:14.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.724+0000 7fd3add7b700 1 -- start start 2024-03-24T03:00:14.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3add7b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3a8101070 0x7fd3a8102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:14.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3add7b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3a8101a30 0x7fd3a8103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:14.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3add7b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3a8107120 0x7fd3a81037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:14.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3add7b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a8111330 con 0x7fd3a8101a30 2024-03-24T03:00:14.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3add7b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a81111b0 con 0x7fd3a8101070 2024-03-24T03:00:14.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3add7b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a81114b0 con 0x7fd3a8107120 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3a6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3a8101a30 0x7fd3a8103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3a6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3a8101a30 0x7fd3a8103260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53040/0 (socket says 172.21.15.17:53040) 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3a6ffd700 1 -- 172.21.15.17:0/3402609311 learned_addr learned my addr 172.21.15.17:0/3402609311 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3a77fe700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3a8101070 0x7fd3a8102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3a7fff700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3a8107120 0x7fd3a81037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3a6ffd700 1 -- 172.21.15.17:0/3402609311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3a8107120 msgr2=0x7fd3a81037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3a6ffd700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3a8107120 0x7fd3a81037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3a6ffd700 1 -- 172.21.15.17:0/3402609311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3a8101070 msgr2=0x7fd3a8102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3a6ffd700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3a8101070 0x7fd3a8102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3a6ffd700 1 -- 172.21.15.17:0/3402609311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a81a6e00 con 0x7fd3a8101a30 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.725+0000 7fd3a6ffd700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3a8101a30 0x7fd3a8103260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00bae0 tx=0x7fd39c00bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.726+0000 7fd3a4ff9700 1 -- 172.21.15.17:0/3402609311 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c026070 con 0x7fd3a8101a30 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.726+0000 7fd3a4ff9700 1 -- 172.21.15.17:0/3402609311 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd39c003d70 con 0x7fd3a8101a30 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.726+0000 7fd3add7b700 1 -- 172.21.15.17:0/3402609311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a81a7090 con 0x7fd3a8101a30 2024-03-24T03:00:14.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.726+0000 7fd3a4ff9700 1 -- 172.21.15.17:0/3402609311 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c021410 con 0x7fd3a8101a30 2024-03-24T03:00:14.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.726+0000 7fd3add7b700 1 -- 172.21.15.17:0/3402609311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a81a7570 con 0x7fd3a8101a30 2024-03-24T03:00:14.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.727+0000 7fd3add7b700 1 -- 172.21.15.17:0/3402609311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a8068e50 con 0x7fd3a8101a30 2024-03-24T03:00:14.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.728+0000 7fd3a4ff9700 1 -- 172.21.15.17:0/3402609311 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd39c029070 con 0x7fd3a8101a30 2024-03-24T03:00:14.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.729+0000 7fd3a4ff9700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd3900a2890 0x7fd3900a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:14.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.729+0000 7fd3a77fe700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd3900a2890 0x7fd3900a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:14.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.729+0000 7fd3a4ff9700 1 -- 172.21.15.17:0/3402609311 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fd39c0d0300 con 0x7fd3a8101a30 2024-03-24T03:00:14.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.729+0000 7fd3a77fe700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd3900a2890 0x7fd3900a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd398009be0 tx=0x7fd398009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:14.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.733+0000 7fd3a4ff9700 1 -- 172.21.15.17:0/3402609311 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd39c013070 con 0x7fd3a8101a30 2024-03-24T03:00:14.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.932+0000 7fd3add7b700 1 -- 172.21.15.17:0/3402609311 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd3a8105e70 con 0x7fd3900a2890 2024-03-24T03:00:14.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.941+0000 7fd3a4ff9700 1 -- 172.21.15.17:0/3402609311 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340515 (secure 0 0 0) 0x7fd3a8105e70 con 0x7fd3900a2890 2024-03-24T03:00:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.944+0000 7fd3add7b700 1 -- 172.21.15.17:0/3402609311 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd3900a2890 msgr2=0x7fd3900a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.944+0000 7fd3add7b700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd3900a2890 0x7fd3900a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd398009be0 tx=0x7fd398009260 comp rx=0 tx=0).stop 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.944+0000 7fd3add7b700 1 -- 172.21.15.17:0/3402609311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3a8101a30 msgr2=0x7fd3a8103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.944+0000 7fd3add7b700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3a8101a30 0x7fd3a8103260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00bae0 tx=0x7fd39c00bb10 comp rx=0 tx=0).stop 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.944+0000 7fd3add7b700 1 -- 172.21.15.17:0/3402609311 shutdown_connections 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.944+0000 7fd3add7b700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd3900a2890 0x7fd3900a4d50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.944+0000 7fd3add7b700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3a8107120 0x7fd3a81037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.944+0000 7fd3add7b700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3a8101070 0x7fd3a8102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.944+0000 7fd3add7b700 1 --2- 172.21.15.17:0/3402609311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3a8101a30 0x7fd3a8103260 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.944+0000 7fd3add7b700 1 -- 172.21.15.17:0/3402609311 >> 172.21.15.17:0/3402609311 conn(0x7fd3a80780a0 msgr2=0x7fd3a806aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.944+0000 7fd3add7b700 1 -- 172.21.15.17:0/3402609311 shutdown_connections 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:14.944+0000 7fd3add7b700 1 -- 172.21.15.17:0/3402609311 wait complete. 2024-03-24T03:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-24T03:00:15.679 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-03-24T03:00:13.972205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665081,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":908,"num_read_kb":590,"num_write":432,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":413324,"kb_used_data":14252,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124528500,"statfs":{"total":1151940427776,"available":1151517184000,"internally_reserved":0,"allocated":14594048,"data_stored":7979345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":408532281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":73,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003675"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.999713+0000","last_change":"2024-03-24T02:58:36.978486+0000","last_active":"2024-03-24T02:59:59.999713+0000","last_peered":"2024-03-24T02:59:59.999713+0000","last_clean":"2024-03-24T02:59:59.999713+0000","last_became_active":"2024-03-24T02:58:36.976937+0000","last_became_peered":"2024-03-24T02:58:36.976937+0000","last_unstale":"2024-03-24T02:59:59.999713+0000","last_undegraded":"2024-03-24T02:59:59.999713+0000","last_fullsized":"2024-03-24T02:59:59.999713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:25:09.439179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.994756+0000","last_change":"2024-03-24T02:58:36.983050+0000","last_active":"2024-03-24T02:59:59.994756+0000","last_peered":"2024-03-24T02:59:59.994756+0000","last_clean":"2024-03-24T02:59:59.994756+0000","last_became_active":"2024-03-24T02:58:36.982741+0000","last_became_peered":"2024-03-24T02:58:36.982741+0000","last_unstale":"2024-03-24T02:59:59.994756+0000","last_undegraded":"2024-03-24T02:59:59.994756+0000","last_fullsized":"2024-03-24T02:59:59.994756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:26:35.938559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.999873+0000","last_change":"2024-03-24T02:58:36.976313+0000","last_active":"2024-03-24T02:59:59.999873+0000","last_peered":"2024-03-24T02:59:59.999873+0000","last_clean":"2024-03-24T02:59:59.999873+0000","last_became_active":"2024-03-24T02:58:36.975087+0000","last_became_peered":"2024-03-24T02:58:36.975087+0000","last_unstale":"2024-03-24T02:59:59.999873+0000","last_undegraded":"2024-03-24T02:59:59.999873+0000","last_fullsized":"2024-03-24T02:59:59.999873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:22:31.476824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995743+0000","last_change":"2024-03-24T02:58:36.977632+0000","last_active":"2024-03-24T02:59:59.995743+0000","last_peered":"2024-03-24T02:59:59.995743+0000","last_clean":"2024-03-24T02:59:59.995743+0000","last_became_active":"2024-03-24T02:58:36.977111+0000","last_became_peered":"2024-03-24T02:58:36.977111+0000","last_unstale":"2024-03-24T02:59:59.995743+0000","last_undegraded":"2024-03-24T02:59:59.995743+0000","last_fullsized":"2024-03-24T02:59:59.995743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:47:07.874538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.990749+0000","last_change":"2024-03-24T02:58:36.967543+0000","last_active":"2024-03-24T02:59:59.990749+0000","last_peered":"2024-03-24T02:59:59.990749+0000","last_clean":"2024-03-24T02:59:59.990749+0000","last_became_active":"2024-03-24T02:58:36.967108+0000","last_became_peered":"2024-03-24T02:58:36.967108+0000","last_unstale":"2024-03-24T02:59:59.990749+0000","last_undegraded":"2024-03-24T02:59:59.990749+0000","last_fullsized":"2024-03-24T02:59:59.990749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:38:46.280891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.980432+0000","last_change":"2024-03-24T02:58:36.977777+0000","last_active":"2024-03-24T02:59:59.980432+0000","last_peered":"2024-03-24T02:59:59.980432+0000","last_clean":"2024-03-24T02:59:59.980432+0000","last_became_active":"2024-03-24T02:58:36.976615+0000","last_became_peered":"2024-03-24T02:58:36.976615+0000","last_unstale":"2024-03-24T02:59:59.980432+0000","last_undegraded":"2024-03-24T02:59:59.980432+0000","last_fullsized":"2024-03-24T02:59:59.980432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:13:15.149963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995932+0000","last_change":"2024-03-24T02:58:36.990832+0000","last_active":"2024-03-24T02:59:59.995932+0000","last_peered":"2024-03-24T02:59:59.995932+0000","last_clean":"2024-03-24T02:59:59.995932+0000","last_became_active":"2024-03-24T02:58:36.990395+0000","last_became_peered":"2024-03-24T02:58:36.990395+0000","last_unstale":"2024-03-24T02:59:59.995932+0000","last_undegraded":"2024-03-24T02:59:59.995932+0000","last_fullsized":"2024-03-24T02:59:59.995932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:34.915192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.982699+0000","last_change":"2024-03-24T02:59:31.132828+0000","last_active":"2024-03-24T02:59:59.982699+0000","last_peered":"2024-03-24T02:59:59.982699+0000","last_clean":"2024-03-24T02:59:59.982699+0000","last_became_active":"2024-03-24T02:59:31.132439+0000","last_became_peered":"2024-03-24T02:59:31.132439+0000","last_unstale":"2024-03-24T02:59:59.982699+0000","last_undegraded":"2024-03-24T02:59:59.982699+0000","last_fullsized":"2024-03-24T02:59:59.982699+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:06:57.643149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,10],"acting":[9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:00.000631+0000","last_change":"2024-03-24T02:58:36.989173+0000","last_active":"2024-03-24T03:00:00.000631+0000","last_peered":"2024-03-24T03:00:00.000631+0000","last_clean":"2024-03-24T03:00:00.000631+0000","last_became_active":"2024-03-24T02:58:36.988771+0000","last_became_peered":"2024-03-24T02:58:36.988771+0000","last_unstale":"2024-03-24T03:00:00.000631+0000","last_undegraded":"2024-03-24T03:00:00.000631+0000","last_fullsized":"2024-03-24T03:00:00.000631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:51:01.790952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.989886+0000","last_change":"2024-03-24T02:58:36.978505+0000","last_active":"2024-03-24T02:59:59.989886+0000","last_peered":"2024-03-24T02:59:59.989886+0000","last_clean":"2024-03-24T02:59:59.989886+0000","last_became_active":"2024-03-24T02:58:36.978145+0000","last_became_peered":"2024-03-24T02:58:36.978145+0000","last_unstale":"2024-03-24T02:59:59.989886+0000","last_undegraded":"2024-03-24T02:59:59.989886+0000","last_fullsized":"2024-03-24T02:59:59.989886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:06:41.874675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.975357+0000","last_change":"2024-03-24T02:59:31.616102+0000","last_active":"2024-03-24T02:59:59.975357+0000","last_peered":"2024-03-24T02:59:59.975357+0000","last_clean":"2024-03-24T02:59:59.975357+0000","last_became_active":"2024-03-24T02:59:31.615741+0000","last_became_peered":"2024-03-24T02:59:31.615741+0000","last_unstale":"2024-03-24T02:59:59.975357+0000","last_undegraded":"2024-03-24T02:59:59.975357+0000","last_fullsized":"2024-03-24T02:59:59.975357+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:04:55.547648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,9],"acting":[7,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.985399+0000","last_change":"2024-03-24T02:58:36.979852+0000","last_active":"2024-03-24T02:59:59.985399+0000","last_peered":"2024-03-24T02:59:59.985399+0000","last_clean":"2024-03-24T02:59:59.985399+0000","last_became_active":"2024-03-24T02:58:36.979506+0000","last_became_peered":"2024-03-24T02:58:36.979506+0000","last_unstale":"2024-03-24T02:59:59.985399+0000","last_undegraded":"2024-03-24T02:59:59.985399+0000","last_fullsized":"2024-03-24T02:59:59.985399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:01:27.493694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.996654+0000","last_change":"2024-03-24T02:58:36.977018+0000","last_active":"2024-03-24T02:59:59.996654+0000","last_peered":"2024-03-24T02:59:59.996654+0000","last_clean":"2024-03-24T02:59:59.996654+0000","last_became_active":"2024-03-24T02:58:36.976669+0000","last_became_peered":"2024-03-24T02:58:36.976669+0000","last_unstale":"2024-03-24T02:59:59.996654+0000","last_undegraded":"2024-03-24T02:59:59.996654+0000","last_fullsized":"2024-03-24T02:59:59.996654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:23:15.049093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995265+0000","last_change":"2024-03-24T02:58:36.973200+0000","last_active":"2024-03-24T02:59:59.995265+0000","last_peered":"2024-03-24T02:59:59.995265+0000","last_clean":"2024-03-24T02:59:59.995265+0000","last_became_active":"2024-03-24T02:58:36.972828+0000","last_became_peered":"2024-03-24T02:58:36.972828+0000","last_unstale":"2024-03-24T02:59:59.995265+0000","last_undegraded":"2024-03-24T02:59:59.995265+0000","last_fullsized":"2024-03-24T02:59:59.995265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:42:25.122881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.994823+0000","last_change":"2024-03-24T02:58:36.977686+0000","last_active":"2024-03-24T02:59:59.994823+0000","last_peered":"2024-03-24T02:59:59.994823+0000","last_clean":"2024-03-24T02:59:59.994823+0000","last_became_active":"2024-03-24T02:58:36.976311+0000","last_became_peered":"2024-03-24T02:58:36.976311+0000","last_unstale":"2024-03-24T02:59:59.994823+0000","last_undegraded":"2024-03-24T02:59:59.994823+0000","last_fullsized":"2024-03-24T02:59:59.994823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:10:04.585726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.999935+0000","last_change":"2024-03-24T02:58:36.985336+0000","last_active":"2024-03-24T02:59:59.999935+0000","last_peered":"2024-03-24T02:59:59.999935+0000","last_clean":"2024-03-24T02:59:59.999935+0000","last_became_active":"2024-03-24T02:58:36.984842+0000","last_became_peered":"2024-03-24T02:58:36.984842+0000","last_unstale":"2024-03-24T02:59:59.999935+0000","last_undegraded":"2024-03-24T02:59:59.999935+0000","last_fullsized":"2024-03-24T02:59:59.999935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:52:41.819132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:00.000107+0000","last_change":"2024-03-24T02:58:36.978310+0000","last_active":"2024-03-24T03:00:00.000107+0000","last_peered":"2024-03-24T03:00:00.000107+0000","last_clean":"2024-03-24T03:00:00.000107+0000","last_became_active":"2024-03-24T02:58:36.977396+0000","last_became_peered":"2024-03-24T02:58:36.977396+0000","last_unstale":"2024-03-24T03:00:00.000107+0000","last_undegraded":"2024-03-24T03:00:00.000107+0000","last_fullsized":"2024-03-24T03:00:00.000107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:37:21.200591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.972767+0000","last_change":"2024-03-24T02:58:36.979514+0000","last_active":"2024-03-24T02:59:59.972767+0000","last_peered":"2024-03-24T02:59:59.972767+0000","last_clean":"2024-03-24T02:59:59.972767+0000","last_became_active":"2024-03-24T02:58:36.979155+0000","last_became_peered":"2024-03-24T02:58:36.979155+0000","last_unstale":"2024-03-24T02:59:59.972767+0000","last_undegraded":"2024-03-24T02:59:59.972767+0000","last_fullsized":"2024-03-24T02:59:59.972767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:49:37.142195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.984643+0000","last_change":"2024-03-24T02:58:36.971546+0000","last_active":"2024-03-24T02:59:59.984643+0000","last_peered":"2024-03-24T02:59:59.984643+0000","last_clean":"2024-03-24T02:59:59.984643+0000","last_became_active":"2024-03-24T02:58:36.971210+0000","last_became_peered":"2024-03-24T02:58:36.971210+0000","last_unstale":"2024-03-24T02:59:59.984643+0000","last_undegraded":"2024-03-24T02:59:59.984643+0000","last_fullsized":"2024-03-24T02:59:59.984643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:35:21.274898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.969927+0000","last_change":"2024-03-24T02:58:40.994331+0000","last_active":"2024-03-24T02:59:30.969927+0000","last_peered":"2024-03-24T02:59:30.969927+0000","last_clean":"2024-03-24T02:59:30.969927+0000","last_became_active":"2024-03-24T02:58:40.993880+0000","last_became_peered":"2024-03-24T02:58:40.993880+0000","last_unstale":"2024-03-24T02:59:30.969927+0000","last_undegraded":"2024-03-24T02:59:30.969927+0000","last_fullsized":"2024-03-24T02:59:30.969927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:58:46.876872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:10.172092+0000","last_change":"2024-03-24T02:58:36.132975+0000","last_active":"2024-03-24T03:00:10.172092+0000","last_peered":"2024-03-24T03:00:10.172092+0000","last_clean":"2024-03-24T03:00:10.172092+0000","last_became_active":"2024-03-24T02:58:36.132640+0000","last_became_peered":"2024-03-24T02:58:36.132640+0000","last_unstale":"2024-03-24T03:00:10.172092+0000","last_undegraded":"2024-03-24T03:00:10.172092+0000","last_fullsized":"2024-03-24T03:00:10.172092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:37:03.765773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.983286+0000","last_change":"2024-03-24T02:59:30.973326+0000","last_active":"2024-03-24T02:59:59.983286+0000","last_peered":"2024-03-24T02:59:59.983286+0000","last_clean":"2024-03-24T02:59:59.983286+0000","last_became_active":"2024-03-24T02:59:30.972939+0000","last_became_peered":"2024-03-24T02:59:30.972939+0000","last_unstale":"2024-03-24T02:59:59.983286+0000","last_undegraded":"2024-03-24T02:59:59.983286+0000","last_fullsized":"2024-03-24T02:59:59.983286+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:53:31.582648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970362+0000","last_change":"2024-03-24T02:58:40.997896+0000","last_active":"2024-03-24T02:59:30.970362+0000","last_peered":"2024-03-24T02:59:30.970362+0000","last_clean":"2024-03-24T02:59:30.970362+0000","last_became_active":"2024-03-24T02:58:40.996845+0000","last_became_peered":"2024-03-24T02:58:40.996845+0000","last_unstale":"2024-03-24T02:59:30.970362+0000","last_undegraded":"2024-03-24T02:59:30.970362+0000","last_fullsized":"2024-03-24T02:59:30.970362+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:51:17.031332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.973187+0000","last_change":"2024-03-24T02:58:36.139069+0000","last_active":"2024-03-24T02:59:30.973187+0000","last_peered":"2024-03-24T02:59:30.973187+0000","last_clean":"2024-03-24T02:59:30.973187+0000","last_became_active":"2024-03-24T02:58:36.138571+0000","last_became_peered":"2024-03-24T02:58:36.138571+0000","last_unstale":"2024-03-24T02:59:30.973187+0000","last_undegraded":"2024-03-24T02:59:30.973187+0000","last_fullsized":"2024-03-24T02:59:30.973187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:17:44.510904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.999702+0000","last_change":"2024-03-24T02:58:36.976727+0000","last_active":"2024-03-24T02:59:59.999702+0000","last_peered":"2024-03-24T02:59:59.999702+0000","last_clean":"2024-03-24T02:59:59.999702+0000","last_became_active":"2024-03-24T02:58:36.976076+0000","last_became_peered":"2024-03-24T02:58:36.976076+0000","last_unstale":"2024-03-24T02:59:59.999702+0000","last_undegraded":"2024-03-24T02:59:59.999702+0000","last_fullsized":"2024-03-24T02:59:59.999702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:33:03.490882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970899+0000","last_change":"2024-03-24T02:58:41.003025+0000","last_active":"2024-03-24T02:59:30.970899+0000","last_peered":"2024-03-24T02:59:30.970899+0000","last_clean":"2024-03-24T02:59:30.970899+0000","last_became_active":"2024-03-24T02:58:41.002270+0000","last_became_peered":"2024-03-24T02:58:41.002270+0000","last_unstale":"2024-03-24T02:59:30.970899+0000","last_undegraded":"2024-03-24T02:59:30.970899+0000","last_fullsized":"2024-03-24T02:59:30.970899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:31:36.390191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.615123+0000","last_change":"2024-03-24T02:58:36.135836+0000","last_active":"2024-03-24T02:59:31.615123+0000","last_peered":"2024-03-24T02:59:31.615123+0000","last_clean":"2024-03-24T02:59:31.615123+0000","last_became_active":"2024-03-24T02:58:36.135476+0000","last_became_peered":"2024-03-24T02:58:36.135476+0000","last_unstale":"2024-03-24T02:59:31.615123+0000","last_undegraded":"2024-03-24T02:59:31.615123+0000","last_fullsized":"2024-03-24T02:59:31.615123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:30:14.058373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:00.002118+0000","last_change":"2024-03-24T02:58:36.981600+0000","last_active":"2024-03-24T03:00:00.002118+0000","last_peered":"2024-03-24T03:00:00.002118+0000","last_clean":"2024-03-24T03:00:00.002118+0000","last_became_active":"2024-03-24T02:58:36.981222+0000","last_became_peered":"2024-03-24T02:58:36.981222+0000","last_unstale":"2024-03-24T03:00:00.002118+0000","last_undegraded":"2024-03-24T03:00:00.002118+0000","last_fullsized":"2024-03-24T03:00:00.002118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:11:33.771157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970402+0000","last_change":"2024-03-24T02:58:40.996838+0000","last_active":"2024-03-24T02:59:30.970402+0000","last_peered":"2024-03-24T02:59:30.970402+0000","last_clean":"2024-03-24T02:59:30.970402+0000","last_became_active":"2024-03-24T02:58:40.996320+0000","last_became_peered":"2024-03-24T02:58:40.996320+0000","last_unstale":"2024-03-24T02:59:30.970402+0000","last_undegraded":"2024-03-24T02:59:30.970402+0000","last_fullsized":"2024-03-24T02:59:30.970402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:11:34.124402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.614884+0000","last_change":"2024-03-24T02:58:36.132406+0000","last_active":"2024-03-24T02:59:31.614884+0000","last_peered":"2024-03-24T02:59:31.614884+0000","last_clean":"2024-03-24T02:59:31.614884+0000","last_became_active":"2024-03-24T02:58:36.132044+0000","last_became_peered":"2024-03-24T02:58:36.132044+0000","last_unstale":"2024-03-24T02:59:31.614884+0000","last_undegraded":"2024-03-24T02:59:31.614884+0000","last_fullsized":"2024-03-24T02:59:31.614884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:35:53.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.999956+0000","last_change":"2024-03-24T02:58:36.977989+0000","last_active":"2024-03-24T02:59:59.999956+0000","last_peered":"2024-03-24T02:59:59.999956+0000","last_clean":"2024-03-24T02:59:59.999956+0000","last_became_active":"2024-03-24T02:58:36.977653+0000","last_became_peered":"2024-03-24T02:58:36.977653+0000","last_unstale":"2024-03-24T02:59:59.999956+0000","last_undegraded":"2024-03-24T02:59:59.999956+0000","last_fullsized":"2024-03-24T02:59:59.999956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:21:18.364057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.969630+0000","last_change":"2024-03-24T02:58:40.989082+0000","last_active":"2024-03-24T02:59:30.969630+0000","last_peered":"2024-03-24T02:59:30.969630+0000","last_clean":"2024-03-24T02:59:30.969630+0000","last_became_active":"2024-03-24T02:58:40.988731+0000","last_became_peered":"2024-03-24T02:58:40.988731+0000","last_unstale":"2024-03-24T02:59:30.969630+0000","last_undegraded":"2024-03-24T02:59:30.969630+0000","last_fullsized":"2024-03-24T02:59:30.969630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:34:39.489446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:12.171996+0000","last_change":"2024-03-24T02:58:36.135785+0000","last_active":"2024-03-24T03:00:12.171996+0000","last_peered":"2024-03-24T03:00:12.171996+0000","last_clean":"2024-03-24T03:00:12.171996+0000","last_became_active":"2024-03-24T02:58:36.135470+0000","last_became_peered":"2024-03-24T02:58:36.135470+0000","last_unstale":"2024-03-24T03:00:12.171996+0000","last_undegraded":"2024-03-24T03:00:12.171996+0000","last_fullsized":"2024-03-24T03:00:12.171996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:41:26.529773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.975619+0000","last_change":"2024-03-24T02:58:36.978241+0000","last_active":"2024-03-24T02:59:59.975619+0000","last_peered":"2024-03-24T02:59:59.975619+0000","last_clean":"2024-03-24T02:59:59.975619+0000","last_became_active":"2024-03-24T02:58:36.977879+0000","last_became_peered":"2024-03-24T02:58:36.977879+0000","last_unstale":"2024-03-24T02:59:59.975619+0000","last_undegraded":"2024-03-24T02:59:59.975619+0000","last_fullsized":"2024-03-24T02:59:59.975619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:44:41.190736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.616395+0000","last_change":"2024-03-24T02:59:31.616395+0000","last_active":"2024-03-24T02:59:31.616395+0000","last_peered":"2024-03-24T02:59:31.616395+0000","last_clean":"2024-03-24T02:59:31.616395+0000","last_became_active":"2024-03-24T02:59:31.615925+0000","last_became_peered":"2024-03-24T02:59:31.615925+0000","last_unstale":"2024-03-24T02:59:31.616395+0000","last_undegraded":"2024-03-24T02:59:31.616395+0000","last_fullsized":"2024-03-24T02:59:31.616395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:54:31.269893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.127563+0000","last_change":"2024-03-24T02:58:36.132242+0000","last_active":"2024-03-24T02:59:31.127563+0000","last_peered":"2024-03-24T02:59:31.127563+0000","last_clean":"2024-03-24T02:59:31.127563+0000","last_became_active":"2024-03-24T02:58:36.131792+0000","last_became_peered":"2024-03-24T02:58:36.131792+0000","last_unstale":"2024-03-24T02:59:31.127563+0000","last_undegraded":"2024-03-24T02:59:31.127563+0000","last_fullsized":"2024-03-24T02:59:31.127563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:09:16.751023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.997565+0000","last_change":"2024-03-24T02:58:36.977306+0000","last_active":"2024-03-24T02:59:59.997565+0000","last_peered":"2024-03-24T02:59:59.997565+0000","last_clean":"2024-03-24T02:59:59.997565+0000","last_became_active":"2024-03-24T02:58:36.976955+0000","last_became_peered":"2024-03-24T02:58:36.976955+0000","last_unstale":"2024-03-24T02:59:59.997565+0000","last_undegraded":"2024-03-24T02:59:59.997565+0000","last_fullsized":"2024-03-24T02:59:59.997565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:09:56.568781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.362779+0000","last_change":"2024-03-24T02:58:40.999977+0000","last_active":"2024-03-24T02:59:31.362779+0000","last_peered":"2024-03-24T02:59:31.362779+0000","last_clean":"2024-03-24T02:59:31.362779+0000","last_became_active":"2024-03-24T02:58:40.999564+0000","last_became_peered":"2024-03-24T02:58:40.999564+0000","last_unstale":"2024-03-24T02:59:31.362779+0000","last_undegraded":"2024-03-24T02:59:31.362779+0000","last_fullsized":"2024-03-24T02:59:31.362779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:16:58.022961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.131256+0000","last_change":"2024-03-24T02:58:36.123622+0000","last_active":"2024-03-24T02:59:31.131256+0000","last_peered":"2024-03-24T02:59:31.131256+0000","last_clean":"2024-03-24T02:59:31.131256+0000","last_became_active":"2024-03-24T02:58:36.123221+0000","last_became_peered":"2024-03-24T02:58:36.123221+0000","last_unstale":"2024-03-24T02:59:31.131256+0000","last_undegraded":"2024-03-24T02:59:31.131256+0000","last_fullsized":"2024-03-24T02:59:31.131256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:05:26.781268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.986606+0000","last_change":"2024-03-24T02:58:36.983446+0000","last_active":"2024-03-24T02:59:59.986606+0000","last_peered":"2024-03-24T02:59:59.986606+0000","last_clean":"2024-03-24T02:59:59.986606+0000","last_became_active":"2024-03-24T02:58:36.981401+0000","last_became_peered":"2024-03-24T02:58:36.981401+0000","last_unstale":"2024-03-24T02:59:59.986606+0000","last_undegraded":"2024-03-24T02:59:59.986606+0000","last_fullsized":"2024-03-24T02:59:59.986606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:12:42.836678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.972459+0000","last_change":"2024-03-24T02:58:40.999348+0000","last_active":"2024-03-24T02:59:30.972459+0000","last_peered":"2024-03-24T02:59:30.972459+0000","last_clean":"2024-03-24T02:59:30.972459+0000","last_became_active":"2024-03-24T02:58:40.998932+0000","last_became_peered":"2024-03-24T02:58:40.998932+0000","last_unstale":"2024-03-24T02:59:30.972459+0000","last_undegraded":"2024-03-24T02:59:30.972459+0000","last_fullsized":"2024-03-24T02:59:30.972459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:24:29.294464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971830+0000","last_change":"2024-03-24T02:58:36.137640+0000","last_active":"2024-03-24T02:59:30.971830+0000","last_peered":"2024-03-24T02:59:30.971830+0000","last_clean":"2024-03-24T02:59:30.971830+0000","last_became_active":"2024-03-24T02:58:36.137237+0000","last_became_peered":"2024-03-24T02:58:36.137237+0000","last_unstale":"2024-03-24T02:59:30.971830+0000","last_undegraded":"2024-03-24T02:59:30.971830+0000","last_fullsized":"2024-03-24T02:59:30.971830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:52:52.246968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.982845+0000","last_change":"2024-03-24T02:58:36.971696+0000","last_active":"2024-03-24T02:59:59.982845+0000","last_peered":"2024-03-24T02:59:59.982845+0000","last_clean":"2024-03-24T02:59:59.982845+0000","last_became_active":"2024-03-24T02:58:36.971318+0000","last_became_peered":"2024-03-24T02:58:36.971318+0000","last_unstale":"2024-03-24T02:59:59.982845+0000","last_undegraded":"2024-03-24T02:59:59.982845+0000","last_fullsized":"2024-03-24T02:59:59.982845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:01:30.976284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971021+0000","last_change":"2024-03-24T02:58:40.995887+0000","last_active":"2024-03-24T02:59:30.971021+0000","last_peered":"2024-03-24T02:59:30.971021+0000","last_clean":"2024-03-24T02:59:30.971021+0000","last_became_active":"2024-03-24T02:58:40.995242+0000","last_became_peered":"2024-03-24T02:58:40.995242+0000","last_unstale":"2024-03-24T02:59:30.971021+0000","last_undegraded":"2024-03-24T02:59:30.971021+0000","last_fullsized":"2024-03-24T02:59:30.971021+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:33:40.376374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:12.171523+0000","last_change":"2024-03-24T02:58:36.128252+0000","last_active":"2024-03-24T03:00:12.171523+0000","last_peered":"2024-03-24T03:00:12.171523+0000","last_clean":"2024-03-24T03:00:12.171523+0000","last_became_active":"2024-03-24T02:58:36.127888+0000","last_became_peered":"2024-03-24T02:58:36.127888+0000","last_unstale":"2024-03-24T03:00:12.171523+0000","last_undegraded":"2024-03-24T03:00:12.171523+0000","last_fullsized":"2024-03-24T03:00:12.171523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:31:39.341123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:00.001689+0000","last_change":"2024-03-24T02:58:36.983836+0000","last_active":"2024-03-24T03:00:00.001689+0000","last_peered":"2024-03-24T03:00:00.001689+0000","last_clean":"2024-03-24T03:00:00.001689+0000","last_became_active":"2024-03-24T02:58:36.983481+0000","last_became_peered":"2024-03-24T02:58:36.983481+0000","last_unstale":"2024-03-24T03:00:00.001689+0000","last_undegraded":"2024-03-24T03:00:00.001689+0000","last_fullsized":"2024-03-24T03:00:00.001689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:01:01.707796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.967591+0000","last_change":"2024-03-24T02:58:41.000176+0000","last_active":"2024-03-24T02:59:30.967591+0000","last_peered":"2024-03-24T02:59:30.967591+0000","last_clean":"2024-03-24T02:59:30.967591+0000","last_became_active":"2024-03-24T02:58:40.999561+0000","last_became_peered":"2024-03-24T02:58:40.999561+0000","last_unstale":"2024-03-24T02:59:30.967591+0000","last_undegraded":"2024-03-24T02:59:30.967591+0000","last_fullsized":"2024-03-24T02:59:30.967591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:20:28.678497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.974070+0000","last_change":"2024-03-24T02:58:36.131588+0000","last_active":"2024-03-24T02:59:30.974070+0000","last_peered":"2024-03-24T02:59:30.974070+0000","last_clean":"2024-03-24T02:59:30.974070+0000","last_became_active":"2024-03-24T02:58:36.131251+0000","last_became_peered":"2024-03-24T02:58:36.131251+0000","last_unstale":"2024-03-24T02:59:30.974070+0000","last_undegraded":"2024-03-24T02:59:30.974070+0000","last_fullsized":"2024-03-24T02:59:30.974070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:03:22.978352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995038+0000","last_change":"2024-03-24T02:59:31.616123+0000","last_active":"2024-03-24T02:59:59.995038+0000","last_peered":"2024-03-24T02:59:59.995038+0000","last_clean":"2024-03-24T02:59:59.995038+0000","last_became_active":"2024-03-24T02:59:31.615702+0000","last_became_peered":"2024-03-24T02:59:31.615702+0000","last_unstale":"2024-03-24T02:59:59.995038+0000","last_undegraded":"2024-03-24T02:59:59.995038+0000","last_fullsized":"2024-03-24T02:59:59.995038+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:06:00.354916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,8],"acting":[0,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970734+0000","last_change":"2024-03-24T02:58:41.003427+0000","last_active":"2024-03-24T02:59:30.970734+0000","last_peered":"2024-03-24T02:59:30.970734+0000","last_clean":"2024-03-24T02:59:30.970734+0000","last_became_active":"2024-03-24T02:58:41.002249+0000","last_became_peered":"2024-03-24T02:58:41.002249+0000","last_unstale":"2024-03-24T02:59:30.970734+0000","last_undegraded":"2024-03-24T02:59:30.970734+0000","last_fullsized":"2024-03-24T02:59:30.970734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:39:36.587451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970924+0000","last_change":"2024-03-24T02:58:36.132547+0000","last_active":"2024-03-24T02:59:30.970924+0000","last_peered":"2024-03-24T02:59:30.970924+0000","last_clean":"2024-03-24T02:59:30.970924+0000","last_became_active":"2024-03-24T02:58:36.132144+0000","last_became_peered":"2024-03-24T02:58:36.132144+0000","last_unstale":"2024-03-24T02:59:30.970924+0000","last_undegraded":"2024-03-24T02:59:30.970924+0000","last_fullsized":"2024-03-24T02:59:30.970924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:07:40.078639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:00.001269+0000","last_change":"2024-03-24T02:58:36.980107+0000","last_active":"2024-03-24T03:00:00.001269+0000","last_peered":"2024-03-24T03:00:00.001269+0000","last_clean":"2024-03-24T03:00:00.001269+0000","last_became_active":"2024-03-24T02:58:36.978914+0000","last_became_peered":"2024-03-24T02:58:36.978914+0000","last_unstale":"2024-03-24T03:00:00.001269+0000","last_undegraded":"2024-03-24T03:00:00.001269+0000","last_fullsized":"2024-03-24T03:00:00.001269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:43:36.956326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.967597+0000","last_change":"2024-03-24T02:58:40.996582+0000","last_active":"2024-03-24T02:59:30.967597+0000","last_peered":"2024-03-24T02:59:30.967597+0000","last_clean":"2024-03-24T02:59:30.967597+0000","last_became_active":"2024-03-24T02:58:40.993673+0000","last_became_peered":"2024-03-24T02:58:40.993673+0000","last_unstale":"2024-03-24T02:59:30.967597+0000","last_undegraded":"2024-03-24T02:59:30.967597+0000","last_fullsized":"2024-03-24T02:59:30.967597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:07:34.506740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":221,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:12.171554+0000","last_change":"2024-03-24T02:58:36.130829+0000","last_active":"2024-03-24T03:00:12.171554+0000","last_peered":"2024-03-24T03:00:12.171554+0000","last_clean":"2024-03-24T03:00:12.171554+0000","last_became_active":"2024-03-24T02:58:36.130483+0000","last_became_peered":"2024-03-24T02:58:36.130483+0000","last_unstale":"2024-03-24T03:00:12.171554+0000","last_undegraded":"2024-03-24T03:00:12.171554+0000","last_fullsized":"2024-03-24T03:00:12.171554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:40:32.407363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66993,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":156,"num_read_kb":143,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.978898+0000","last_change":"2024-03-24T02:58:36.984011+0000","last_active":"2024-03-24T02:59:59.978898+0000","last_peered":"2024-03-24T02:59:59.978898+0000","last_clean":"2024-03-24T02:59:59.978898+0000","last_became_active":"2024-03-24T02:58:36.983623+0000","last_became_peered":"2024-03-24T02:58:36.983623+0000","last_unstale":"2024-03-24T02:59:59.978898+0000","last_undegraded":"2024-03-24T02:59:59.978898+0000","last_fullsized":"2024-03-24T02:59:59.978898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:54:35.745458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971627+0000","last_change":"2024-03-24T02:58:40.993361+0000","last_active":"2024-03-24T02:59:30.971627+0000","last_peered":"2024-03-24T02:59:30.971627+0000","last_clean":"2024-03-24T02:59:30.971627+0000","last_became_active":"2024-03-24T02:58:40.992983+0000","last_became_peered":"2024-03-24T02:58:40.992983+0000","last_unstale":"2024-03-24T02:59:30.971627+0000","last_undegraded":"2024-03-24T02:59:30.971627+0000","last_fullsized":"2024-03-24T02:59:30.971627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:04:21.700904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.969985+0000","last_change":"2024-03-24T02:58:36.131101+0000","last_active":"2024-03-24T02:59:30.969985+0000","last_peered":"2024-03-24T02:59:30.969985+0000","last_clean":"2024-03-24T02:59:30.969985+0000","last_became_active":"2024-03-24T02:58:36.130356+0000","last_became_peered":"2024-03-24T02:58:36.130356+0000","last_unstale":"2024-03-24T02:59:30.969985+0000","last_undegraded":"2024-03-24T02:59:30.969985+0000","last_fullsized":"2024-03-24T02:59:30.969985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:16:40.613073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.992264+0000","last_change":"2024-03-24T02:58:36.983955+0000","last_active":"2024-03-24T02:59:59.992264+0000","last_peered":"2024-03-24T02:59:59.992264+0000","last_clean":"2024-03-24T02:59:59.992264+0000","last_became_active":"2024-03-24T02:58:36.983409+0000","last_became_peered":"2024-03-24T02:58:36.983409+0000","last_unstale":"2024-03-24T02:59:59.992264+0000","last_undegraded":"2024-03-24T02:59:59.992264+0000","last_fullsized":"2024-03-24T02:59:59.992264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:24:14.675779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.362637+0000","last_change":"2024-03-24T02:58:41.004609+0000","last_active":"2024-03-24T02:59:31.362637+0000","last_peered":"2024-03-24T02:59:31.362637+0000","last_clean":"2024-03-24T02:59:31.362637+0000","last_became_active":"2024-03-24T02:58:41.004199+0000","last_became_peered":"2024-03-24T02:58:41.004199+0000","last_unstale":"2024-03-24T02:59:31.362637+0000","last_undegraded":"2024-03-24T02:59:31.362637+0000","last_fullsized":"2024-03-24T02:59:31.362637+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:06:49.717152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971339+0000","last_change":"2024-03-24T02:58:36.137678+0000","last_active":"2024-03-24T02:59:30.971339+0000","last_peered":"2024-03-24T02:59:30.971339+0000","last_clean":"2024-03-24T02:59:30.971339+0000","last_became_active":"2024-03-24T02:58:36.137213+0000","last_became_peered":"2024-03-24T02:58:36.137213+0000","last_unstale":"2024-03-24T02:59:30.971339+0000","last_undegraded":"2024-03-24T02:59:30.971339+0000","last_fullsized":"2024-03-24T02:59:30.971339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:27:50.610128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.987518+0000","last_change":"2024-03-24T02:58:36.981976+0000","last_active":"2024-03-24T02:59:59.987518+0000","last_peered":"2024-03-24T02:59:59.987518+0000","last_clean":"2024-03-24T02:59:59.987518+0000","last_became_active":"2024-03-24T02:58:36.981617+0000","last_became_peered":"2024-03-24T02:58:36.981617+0000","last_unstale":"2024-03-24T02:59:59.987518+0000","last_undegraded":"2024-03-24T02:59:59.987518+0000","last_fullsized":"2024-03-24T02:59:59.987518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:39:43.822859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.968298+0000","last_change":"2024-03-24T02:58:41.001497+0000","last_active":"2024-03-24T02:59:30.968298+0000","last_peered":"2024-03-24T02:59:30.968298+0000","last_clean":"2024-03-24T02:59:30.968298+0000","last_became_active":"2024-03-24T02:58:41.001088+0000","last_became_peered":"2024-03-24T02:58:41.001088+0000","last_unstale":"2024-03-24T02:59:30.968298+0000","last_undegraded":"2024-03-24T02:59:30.968298+0000","last_fullsized":"2024-03-24T02:59:30.968298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:26:51.912623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":108,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:08.170282+0000","last_change":"2024-03-24T02:58:36.135245+0000","last_active":"2024-03-24T03:00:08.170282+0000","last_peered":"2024-03-24T03:00:08.170282+0000","last_clean":"2024-03-24T03:00:08.170282+0000","last_became_active":"2024-03-24T02:58:36.134853+0000","last_became_peered":"2024-03-24T02:58:36.134853+0000","last_unstale":"2024-03-24T03:00:08.170282+0000","last_undegraded":"2024-03-24T03:00:08.170282+0000","last_fullsized":"2024-03-24T03:00:08.170282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:50:11.890138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995857+0000","last_change":"2024-03-24T02:58:36.988925+0000","last_active":"2024-03-24T02:59:59.995857+0000","last_peered":"2024-03-24T02:59:59.995857+0000","last_clean":"2024-03-24T02:59:59.995857+0000","last_became_active":"2024-03-24T02:58:36.987201+0000","last_became_peered":"2024-03-24T02:58:36.987201+0000","last_unstale":"2024-03-24T02:59:59.995857+0000","last_undegraded":"2024-03-24T02:59:59.995857+0000","last_fullsized":"2024-03-24T02:59:59.995857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:38:54.277890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.131876+0000","last_change":"2024-03-24T02:58:41.002868+0000","last_active":"2024-03-24T02:59:31.131876+0000","last_peered":"2024-03-24T02:59:31.131876+0000","last_clean":"2024-03-24T02:59:31.131876+0000","last_became_active":"2024-03-24T02:58:41.002436+0000","last_became_peered":"2024-03-24T02:58:41.002436+0000","last_unstale":"2024-03-24T02:59:31.131876+0000","last_undegraded":"2024-03-24T02:59:31.131876+0000","last_fullsized":"2024-03-24T02:59:31.131876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:31:41.870769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.614592+0000","last_change":"2024-03-24T02:58:36.127705+0000","last_active":"2024-03-24T02:59:31.614592+0000","last_peered":"2024-03-24T02:59:31.614592+0000","last_clean":"2024-03-24T02:59:31.614592+0000","last_became_active":"2024-03-24T02:58:36.127359+0000","last_became_peered":"2024-03-24T02:58:36.127359+0000","last_unstale":"2024-03-24T02:59:31.614592+0000","last_undegraded":"2024-03-24T02:59:31.614592+0000","last_fullsized":"2024-03-24T02:59:31.614592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:14:13.504039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.997412+0000","last_change":"2024-03-24T02:58:36.986285+0000","last_active":"2024-03-24T02:59:59.997412+0000","last_peered":"2024-03-24T02:59:59.997412+0000","last_clean":"2024-03-24T02:59:59.997412+0000","last_became_active":"2024-03-24T02:58:36.983690+0000","last_became_peered":"2024-03-24T02:58:36.983690+0000","last_unstale":"2024-03-24T02:59:59.997412+0000","last_undegraded":"2024-03-24T02:59:59.997412+0000","last_fullsized":"2024-03-24T02:59:59.997412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:14:13.903312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970856+0000","last_change":"2024-03-24T02:58:40.994348+0000","last_active":"2024-03-24T02:59:30.970856+0000","last_peered":"2024-03-24T02:59:30.970856+0000","last_clean":"2024-03-24T02:59:30.970856+0000","last_became_active":"2024-03-24T02:58:40.993871+0000","last_became_peered":"2024-03-24T02:58:40.993871+0000","last_unstale":"2024-03-24T02:59:30.970856+0000","last_undegraded":"2024-03-24T02:59:30.970856+0000","last_fullsized":"2024-03-24T02:59:30.970856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:29:29.236204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.615185+0000","last_change":"2024-03-24T02:58:36.130645+0000","last_active":"2024-03-24T02:59:31.615185+0000","last_peered":"2024-03-24T02:59:31.615185+0000","last_clean":"2024-03-24T02:59:31.615185+0000","last_became_active":"2024-03-24T02:58:36.130289+0000","last_became_peered":"2024-03-24T02:58:36.130289+0000","last_unstale":"2024-03-24T02:59:31.615185+0000","last_undegraded":"2024-03-24T02:59:31.615185+0000","last_fullsized":"2024-03-24T02:59:31.615185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:18:04.085684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.984212+0000","last_change":"2024-03-24T02:58:36.980899+0000","last_active":"2024-03-24T02:59:59.984212+0000","last_peered":"2024-03-24T02:59:59.984212+0000","last_clean":"2024-03-24T02:59:59.984212+0000","last_became_active":"2024-03-24T02:58:36.979893+0000","last_became_peered":"2024-03-24T02:58:36.979893+0000","last_unstale":"2024-03-24T02:59:59.984212+0000","last_undegraded":"2024-03-24T02:59:59.984212+0000","last_fullsized":"2024-03-24T02:59:59.984212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:39.208269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.614857+0000","last_change":"2024-03-24T02:58:40.986381+0000","last_active":"2024-03-24T02:59:31.614857+0000","last_peered":"2024-03-24T02:59:31.614857+0000","last_clean":"2024-03-24T02:59:31.614857+0000","last_became_active":"2024-03-24T02:58:40.985539+0000","last_became_peered":"2024-03-24T02:58:40.985539+0000","last_unstale":"2024-03-24T02:59:31.614857+0000","last_undegraded":"2024-03-24T02:59:31.614857+0000","last_fullsized":"2024-03-24T02:59:31.614857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:10:13.593975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.967847+0000","last_change":"2024-03-24T02:58:36.126405+0000","last_active":"2024-03-24T02:59:30.967847+0000","last_peered":"2024-03-24T02:59:30.967847+0000","last_clean":"2024-03-24T02:59:30.967847+0000","last_became_active":"2024-03-24T02:58:36.126055+0000","last_became_peered":"2024-03-24T02:58:36.126055+0000","last_unstale":"2024-03-24T02:59:30.967847+0000","last_undegraded":"2024-03-24T02:59:30.967847+0000","last_fullsized":"2024-03-24T02:59:30.967847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:04:36.647320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.973993+0000","last_change":"2024-03-24T02:58:36.967546+0000","last_active":"2024-03-24T02:59:59.973993+0000","last_peered":"2024-03-24T02:59:59.973993+0000","last_clean":"2024-03-24T02:59:59.973993+0000","last_became_active":"2024-03-24T02:58:36.967158+0000","last_became_peered":"2024-03-24T02:58:36.967158+0000","last_unstale":"2024-03-24T02:59:59.973993+0000","last_undegraded":"2024-03-24T02:59:59.973993+0000","last_fullsized":"2024-03-24T02:59:59.973993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:06:43.500531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.131597+0000","last_change":"2024-03-24T02:58:41.005489+0000","last_active":"2024-03-24T02:59:31.131597+0000","last_peered":"2024-03-24T02:59:31.131597+0000","last_clean":"2024-03-24T02:59:31.131597+0000","last_became_active":"2024-03-24T02:58:41.004948+0000","last_became_peered":"2024-03-24T02:58:41.004948+0000","last_unstale":"2024-03-24T02:59:31.131597+0000","last_undegraded":"2024-03-24T02:59:31.131597+0000","last_fullsized":"2024-03-24T02:59:31.131597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:19:58.161486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:08.170222+0000","last_change":"2024-03-24T02:58:36.136598+0000","last_active":"2024-03-24T03:00:08.170222+0000","last_peered":"2024-03-24T03:00:08.170222+0000","last_clean":"2024-03-24T03:00:08.170222+0000","last_became_active":"2024-03-24T02:58:36.136210+0000","last_became_peered":"2024-03-24T02:58:36.136210+0000","last_unstale":"2024-03-24T03:00:08.170222+0000","last_undegraded":"2024-03-24T03:00:08.170222+0000","last_fullsized":"2024-03-24T03:00:08.170222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:28:51.216582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.979807+0000","last_change":"2024-03-24T02:58:36.976982+0000","last_active":"2024-03-24T02:59:59.979807+0000","last_peered":"2024-03-24T02:59:59.979807+0000","last_clean":"2024-03-24T02:59:59.979807+0000","last_became_active":"2024-03-24T02:58:36.976429+0000","last_became_peered":"2024-03-24T02:58:36.976429+0000","last_unstale":"2024-03-24T02:59:59.979807+0000","last_undegraded":"2024-03-24T02:59:59.979807+0000","last_fullsized":"2024-03-24T02:59:59.979807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:27:37.153099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.131944+0000","last_change":"2024-03-24T02:58:41.003368+0000","last_active":"2024-03-24T02:59:31.131944+0000","last_peered":"2024-03-24T02:59:31.131944+0000","last_clean":"2024-03-24T02:59:31.131944+0000","last_became_active":"2024-03-24T02:58:41.002950+0000","last_became_peered":"2024-03-24T02:58:41.002950+0000","last_unstale":"2024-03-24T02:59:31.131944+0000","last_undegraded":"2024-03-24T02:59:31.131944+0000","last_fullsized":"2024-03-24T02:59:31.131944+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:27:52.410491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.997471+0000","last_change":"2024-03-24T02:58:36.981969+0000","last_active":"2024-03-24T02:59:59.997471+0000","last_peered":"2024-03-24T02:59:59.997471+0000","last_clean":"2024-03-24T02:59:59.997471+0000","last_became_active":"2024-03-24T02:58:36.981535+0000","last_became_peered":"2024-03-24T02:58:36.981535+0000","last_unstale":"2024-03-24T02:59:59.997471+0000","last_undegraded":"2024-03-24T02:59:59.997471+0000","last_fullsized":"2024-03-24T02:59:59.997471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:45:02.572616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:12.173954+0000","last_change":"2024-03-24T02:58:36.130277+0000","last_active":"2024-03-24T03:00:12.173954+0000","last_peered":"2024-03-24T03:00:12.173954+0000","last_clean":"2024-03-24T03:00:12.173954+0000","last_became_active":"2024-03-24T02:58:36.129909+0000","last_became_peered":"2024-03-24T02:58:36.129909+0000","last_unstale":"2024-03-24T03:00:12.173954+0000","last_undegraded":"2024-03-24T03:00:12.173954+0000","last_fullsized":"2024-03-24T03:00:12.173954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:11:30.314752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.973996+0000","last_change":"2024-03-24T02:58:40.991037+0000","last_active":"2024-03-24T02:59:30.973996+0000","last_peered":"2024-03-24T02:59:30.973996+0000","last_clean":"2024-03-24T02:59:30.973996+0000","last_became_active":"2024-03-24T02:58:40.990382+0000","last_became_peered":"2024-03-24T02:58:40.990382+0000","last_unstale":"2024-03-24T02:59:30.973996+0000","last_undegraded":"2024-03-24T02:59:30.973996+0000","last_fullsized":"2024-03-24T02:59:30.973996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:08:00.048972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.996181+0000","last_change":"2024-03-24T02:58:36.976255+0000","last_active":"2024-03-24T02:59:59.996181+0000","last_peered":"2024-03-24T02:59:59.996181+0000","last_clean":"2024-03-24T02:59:59.996181+0000","last_became_active":"2024-03-24T02:58:36.975937+0000","last_became_peered":"2024-03-24T02:58:36.975937+0000","last_unstale":"2024-03-24T02:59:59.996181+0000","last_undegraded":"2024-03-24T02:59:59.996181+0000","last_fullsized":"2024-03-24T02:59:59.996181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:44:11.310205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970639+0000","last_change":"2024-03-24T02:58:36.126466+0000","last_active":"2024-03-24T02:59:30.970639+0000","last_peered":"2024-03-24T02:59:30.970639+0000","last_clean":"2024-03-24T02:59:30.970639+0000","last_became_active":"2024-03-24T02:58:36.125944+0000","last_became_peered":"2024-03-24T02:58:36.125944+0000","last_unstale":"2024-03-24T02:59:30.970639+0000","last_undegraded":"2024-03-24T02:59:30.970639+0000","last_fullsized":"2024-03-24T02:59:30.970639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:37:17.123640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.973356+0000","last_change":"2024-03-24T02:57:20.795020+0000","last_active":"2024-03-24T02:59:30.973356+0000","last_peered":"2024-03-24T02:59:30.973356+0000","last_clean":"2024-03-24T02:59:30.973356+0000","last_became_active":"2024-03-24T02:57:20.794535+0000","last_became_peered":"2024-03-24T02:57:20.794535+0000","last_unstale":"2024-03-24T02:59:30.973356+0000","last_undegraded":"2024-03-24T02:59:30.973356+0000","last_fullsized":"2024-03-24T02:59:30.973356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:53:54.827889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:53:54.827889+0000","last_clean_scrub_stamp":"2024-03-24T02:53:54.827889+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:47:16.501404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970983+0000","last_change":"2024-03-24T02:58:41.002766+0000","last_active":"2024-03-24T02:59:30.970983+0000","last_peered":"2024-03-24T02:59:30.970983+0000","last_clean":"2024-03-24T02:59:30.970983+0000","last_became_active":"2024-03-24T02:58:41.002401+0000","last_became_peered":"2024-03-24T02:58:41.002401+0000","last_unstale":"2024-03-24T02:59:30.970983+0000","last_undegraded":"2024-03-24T02:59:30.970983+0000","last_fullsized":"2024-03-24T02:59:30.970983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:58:31.401629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.990473+0000","last_change":"2024-03-24T02:58:36.978649+0000","last_active":"2024-03-24T02:59:59.990473+0000","last_peered":"2024-03-24T02:59:59.990473+0000","last_clean":"2024-03-24T02:59:59.990473+0000","last_became_active":"2024-03-24T02:58:36.978136+0000","last_became_peered":"2024-03-24T02:58:36.978136+0000","last_unstale":"2024-03-24T02:59:59.990473+0000","last_undegraded":"2024-03-24T02:59:59.990473+0000","last_fullsized":"2024-03-24T02:59:59.990473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:43:55.598250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.973361+0000","last_change":"2024-03-24T02:58:36.138886+0000","last_active":"2024-03-24T02:59:30.973361+0000","last_peered":"2024-03-24T02:59:30.973361+0000","last_clean":"2024-03-24T02:59:30.973361+0000","last_became_active":"2024-03-24T02:58:36.138582+0000","last_became_peered":"2024-03-24T02:58:36.138582+0000","last_unstale":"2024-03-24T02:59:30.973361+0000","last_undegraded":"2024-03-24T02:59:30.973361+0000","last_fullsized":"2024-03-24T02:59:30.973361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:17:48.129273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.362535+0000","last_change":"2024-03-24T02:58:41.000477+0000","last_active":"2024-03-24T02:59:31.362535+0000","last_peered":"2024-03-24T02:59:31.362535+0000","last_clean":"2024-03-24T02:59:31.362535+0000","last_became_active":"2024-03-24T02:58:41.000157+0000","last_became_peered":"2024-03-24T02:58:41.000157+0000","last_unstale":"2024-03-24T02:59:31.362535+0000","last_undegraded":"2024-03-24T02:59:31.362535+0000","last_fullsized":"2024-03-24T02:59:31.362535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:55:44.870347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.614937+0000","last_change":"2024-03-24T02:58:36.135524+0000","last_active":"2024-03-24T02:59:31.614937+0000","last_peered":"2024-03-24T02:59:31.614937+0000","last_clean":"2024-03-24T02:59:31.614937+0000","last_became_active":"2024-03-24T02:58:36.135170+0000","last_became_peered":"2024-03-24T02:58:36.135170+0000","last_unstale":"2024-03-24T02:59:31.614937+0000","last_undegraded":"2024-03-24T02:59:31.614937+0000","last_fullsized":"2024-03-24T02:59:31.614937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:56:52.737193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.990921+0000","last_change":"2024-03-24T02:58:36.969887+0000","last_active":"2024-03-24T02:59:59.990921+0000","last_peered":"2024-03-24T02:59:59.990921+0000","last_clean":"2024-03-24T02:59:59.990921+0000","last_became_active":"2024-03-24T02:58:36.969510+0000","last_became_peered":"2024-03-24T02:58:36.969510+0000","last_unstale":"2024-03-24T02:59:59.990921+0000","last_undegraded":"2024-03-24T02:59:59.990921+0000","last_fullsized":"2024-03-24T02:59:59.990921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:55:58.135452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970711+0000","last_change":"2024-03-24T02:58:40.996393+0000","last_active":"2024-03-24T02:59:30.970711+0000","last_peered":"2024-03-24T02:59:30.970711+0000","last_clean":"2024-03-24T02:59:30.970711+0000","last_became_active":"2024-03-24T02:58:40.995967+0000","last_became_peered":"2024-03-24T02:58:40.995967+0000","last_unstale":"2024-03-24T02:59:30.970711+0000","last_undegraded":"2024-03-24T02:59:30.970711+0000","last_fullsized":"2024-03-24T02:59:30.970711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:14:51.160730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.968113+0000","last_change":"2024-03-24T02:58:36.135033+0000","last_active":"2024-03-24T02:59:30.968113+0000","last_peered":"2024-03-24T02:59:30.968113+0000","last_clean":"2024-03-24T02:59:30.968113+0000","last_became_active":"2024-03-24T02:58:36.134713+0000","last_became_peered":"2024-03-24T02:58:36.134713+0000","last_unstale":"2024-03-24T02:59:30.968113+0000","last_undegraded":"2024-03-24T02:59:30.968113+0000","last_fullsized":"2024-03-24T02:59:30.968113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:05:55.777016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.993307+0000","last_change":"2024-03-24T02:58:36.984024+0000","last_active":"2024-03-24T02:59:59.993307+0000","last_peered":"2024-03-24T02:59:59.993307+0000","last_clean":"2024-03-24T02:59:59.993307+0000","last_became_active":"2024-03-24T02:58:36.983659+0000","last_became_peered":"2024-03-24T02:58:36.983659+0000","last_unstale":"2024-03-24T02:59:59.993307+0000","last_undegraded":"2024-03-24T02:59:59.993307+0000","last_fullsized":"2024-03-24T02:59:59.993307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:47:59.900749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.969725+0000","last_change":"2024-03-24T02:58:40.991181+0000","last_active":"2024-03-24T02:59:30.969725+0000","last_peered":"2024-03-24T02:59:30.969725+0000","last_clean":"2024-03-24T02:59:30.969725+0000","last_became_active":"2024-03-24T02:58:40.990755+0000","last_became_peered":"2024-03-24T02:58:40.990755+0000","last_unstale":"2024-03-24T02:59:30.969725+0000","last_undegraded":"2024-03-24T02:59:30.969725+0000","last_fullsized":"2024-03-24T02:59:30.969725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:57:05.480364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971003+0000","last_change":"2024-03-24T02:58:36.123100+0000","last_active":"2024-03-24T02:59:30.971003+0000","last_peered":"2024-03-24T02:59:30.971003+0000","last_clean":"2024-03-24T02:59:30.971003+0000","last_became_active":"2024-03-24T02:58:36.122613+0000","last_became_peered":"2024-03-24T02:58:36.122613+0000","last_unstale":"2024-03-24T02:59:30.971003+0000","last_undegraded":"2024-03-24T02:59:30.971003+0000","last_fullsized":"2024-03-24T02:59:30.971003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:31:23.954532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.997313+0000","last_change":"2024-03-24T02:58:36.977010+0000","last_active":"2024-03-24T02:59:59.997313+0000","last_peered":"2024-03-24T02:59:59.997313+0000","last_clean":"2024-03-24T02:59:59.997313+0000","last_became_active":"2024-03-24T02:58:36.976499+0000","last_became_peered":"2024-03-24T02:58:36.976499+0000","last_unstale":"2024-03-24T02:59:59.997313+0000","last_undegraded":"2024-03-24T02:59:59.997313+0000","last_fullsized":"2024-03-24T02:59:59.997313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:18:33.754474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971114+0000","last_change":"2024-03-24T02:58:40.991043+0000","last_active":"2024-03-24T02:59:30.971114+0000","last_peered":"2024-03-24T02:59:30.971114+0000","last_clean":"2024-03-24T02:59:30.971114+0000","last_became_active":"2024-03-24T02:58:40.990549+0000","last_became_peered":"2024-03-24T02:58:40.990549+0000","last_unstale":"2024-03-24T02:59:30.971114+0000","last_undegraded":"2024-03-24T02:59:30.971114+0000","last_fullsized":"2024-03-24T02:59:30.971114+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:20:34.014398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.436877+0000","last_change":"2024-03-24T02:58:36.127420+0000","last_active":"2024-03-24T02:59:31.436877+0000","last_peered":"2024-03-24T02:59:31.436877+0000","last_clean":"2024-03-24T02:59:31.436877+0000","last_became_active":"2024-03-24T02:58:36.126995+0000","last_became_peered":"2024-03-24T02:58:36.126995+0000","last_unstale":"2024-03-24T02:59:31.436877+0000","last_undegraded":"2024-03-24T02:59:31.436877+0000","last_fullsized":"2024-03-24T02:59:31.436877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:01:52.250569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.955504+0000","last_change":"2024-03-24T02:58:36.971978+0000","last_active":"2024-03-24T02:59:59.955504+0000","last_peered":"2024-03-24T02:59:59.955504+0000","last_clean":"2024-03-24T02:59:59.955504+0000","last_became_active":"2024-03-24T02:58:36.969812+0000","last_became_peered":"2024-03-24T02:58:36.969812+0000","last_unstale":"2024-03-24T02:59:59.955504+0000","last_undegraded":"2024-03-24T02:59:59.955504+0000","last_fullsized":"2024-03-24T02:59:59.955504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:44:38.573916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.968435+0000","last_change":"2024-03-24T02:58:40.994515+0000","last_active":"2024-03-24T02:59:30.968435+0000","last_peered":"2024-03-24T02:59:30.968435+0000","last_clean":"2024-03-24T02:59:30.968435+0000","last_became_active":"2024-03-24T02:58:40.994113+0000","last_became_peered":"2024-03-24T02:58:40.994113+0000","last_unstale":"2024-03-24T02:59:30.968435+0000","last_undegraded":"2024-03-24T02:59:30.968435+0000","last_fullsized":"2024-03-24T02:59:30.968435+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:21:58.972660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971131+0000","last_change":"2024-03-24T02:58:36.136590+0000","last_active":"2024-03-24T02:59:30.971131+0000","last_peered":"2024-03-24T02:59:30.971131+0000","last_clean":"2024-03-24T02:59:30.971131+0000","last_became_active":"2024-03-24T02:58:36.136248+0000","last_became_peered":"2024-03-24T02:58:36.136248+0000","last_unstale":"2024-03-24T02:59:30.971131+0000","last_undegraded":"2024-03-24T02:59:30.971131+0000","last_fullsized":"2024-03-24T02:59:30.971131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:20:50.738185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.994480+0000","last_change":"2024-03-24T02:58:36.977145+0000","last_active":"2024-03-24T02:59:59.994480+0000","last_peered":"2024-03-24T02:59:59.994480+0000","last_clean":"2024-03-24T02:59:59.994480+0000","last_became_active":"2024-03-24T02:58:36.975835+0000","last_became_peered":"2024-03-24T02:58:36.975835+0000","last_unstale":"2024-03-24T02:59:59.994480+0000","last_undegraded":"2024-03-24T02:59:59.994480+0000","last_fullsized":"2024-03-24T02:59:59.994480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:33:30.150675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970576+0000","last_change":"2024-03-24T02:58:41.005505+0000","last_active":"2024-03-24T02:59:30.970576+0000","last_peered":"2024-03-24T02:59:30.970576+0000","last_clean":"2024-03-24T02:59:30.970576+0000","last_became_active":"2024-03-24T02:58:41.005063+0000","last_became_peered":"2024-03-24T02:58:41.005063+0000","last_unstale":"2024-03-24T02:59:30.970576+0000","last_undegraded":"2024-03-24T02:59:30.970576+0000","last_fullsized":"2024-03-24T02:59:30.970576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:22:31.947209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.969661+0000","last_change":"2024-03-24T02:58:36.130936+0000","last_active":"2024-03-24T02:59:30.969661+0000","last_peered":"2024-03-24T02:59:30.969661+0000","last_clean":"2024-03-24T02:59:30.969661+0000","last_became_active":"2024-03-24T02:58:36.130581+0000","last_became_peered":"2024-03-24T02:58:36.130581+0000","last_unstale":"2024-03-24T02:59:30.969661+0000","last_undegraded":"2024-03-24T02:59:30.969661+0000","last_fullsized":"2024-03-24T02:59:30.969661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:52.958392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.981649+0000","last_change":"2024-03-24T02:58:36.977062+0000","last_active":"2024-03-24T02:59:59.981649+0000","last_peered":"2024-03-24T02:59:59.981649+0000","last_clean":"2024-03-24T02:59:59.981649+0000","last_became_active":"2024-03-24T02:58:36.976675+0000","last_became_peered":"2024-03-24T02:58:36.976675+0000","last_unstale":"2024-03-24T02:59:59.981649+0000","last_undegraded":"2024-03-24T02:59:59.981649+0000","last_fullsized":"2024-03-24T02:59:59.981649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:37:31.955379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970801+0000","last_change":"2024-03-24T02:58:40.996227+0000","last_active":"2024-03-24T02:59:30.970801+0000","last_peered":"2024-03-24T02:59:30.970801+0000","last_clean":"2024-03-24T02:59:30.970801+0000","last_became_active":"2024-03-24T02:58:40.995394+0000","last_became_peered":"2024-03-24T02:58:40.995394+0000","last_unstale":"2024-03-24T02:59:30.970801+0000","last_undegraded":"2024-03-24T02:59:30.970801+0000","last_fullsized":"2024-03-24T02:59:30.970801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:00:04.370712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.614783+0000","last_change":"2024-03-24T02:58:36.135777+0000","last_active":"2024-03-24T02:59:31.614783+0000","last_peered":"2024-03-24T02:59:31.614783+0000","last_clean":"2024-03-24T02:59:31.614783+0000","last_became_active":"2024-03-24T02:58:36.135398+0000","last_became_peered":"2024-03-24T02:58:36.135398+0000","last_unstale":"2024-03-24T02:59:31.614783+0000","last_undegraded":"2024-03-24T02:59:31.614783+0000","last_fullsized":"2024-03-24T02:59:31.614783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:11:01.641323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.985294+0000","last_change":"2024-03-24T02:58:36.981004+0000","last_active":"2024-03-24T02:59:59.985294+0000","last_peered":"2024-03-24T02:59:59.985294+0000","last_clean":"2024-03-24T02:59:59.985294+0000","last_became_active":"2024-03-24T02:58:36.980258+0000","last_became_peered":"2024-03-24T02:58:36.980258+0000","last_unstale":"2024-03-24T02:59:59.985294+0000","last_undegraded":"2024-03-24T02:59:59.985294+0000","last_fullsized":"2024-03-24T02:59:59.985294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:21:22.479122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.436837+0000","last_change":"2024-03-24T02:58:40.994636+0000","last_active":"2024-03-24T02:59:31.436837+0000","last_peered":"2024-03-24T02:59:31.436837+0000","last_clean":"2024-03-24T02:59:31.436837+0000","last_became_active":"2024-03-24T02:58:40.994122+0000","last_became_peered":"2024-03-24T02:58:40.994122+0000","last_unstale":"2024-03-24T02:59:31.436837+0000","last_undegraded":"2024-03-24T02:59:31.436837+0000","last_fullsized":"2024-03-24T02:59:31.436837+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:55:56.707430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":108,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:08.170254+0000","last_change":"2024-03-24T02:58:36.134528+0000","last_active":"2024-03-24T03:00:08.170254+0000","last_peered":"2024-03-24T03:00:08.170254+0000","last_clean":"2024-03-24T03:00:08.170254+0000","last_became_active":"2024-03-24T02:58:36.134158+0000","last_became_peered":"2024-03-24T02:58:36.134158+0000","last_unstale":"2024-03-24T03:00:08.170254+0000","last_undegraded":"2024-03-24T03:00:08.170254+0000","last_fullsized":"2024-03-24T03:00:08.170254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:50:31.582575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.985991+0000","last_change":"2024-03-24T02:58:36.984545+0000","last_active":"2024-03-24T02:59:59.985991+0000","last_peered":"2024-03-24T02:59:59.985991+0000","last_clean":"2024-03-24T02:59:59.985991+0000","last_became_active":"2024-03-24T02:58:36.982843+0000","last_became_peered":"2024-03-24T02:58:36.982843+0000","last_unstale":"2024-03-24T02:59:59.985991+0000","last_undegraded":"2024-03-24T02:59:59.985991+0000","last_fullsized":"2024-03-24T02:59:59.985991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:50:04.481097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.970923+0000","last_change":"2024-03-24T02:58:41.000190+0000","last_active":"2024-03-24T02:59:30.970923+0000","last_peered":"2024-03-24T02:59:30.970923+0000","last_clean":"2024-03-24T02:59:30.970923+0000","last_became_active":"2024-03-24T02:58:40.999767+0000","last_became_peered":"2024-03-24T02:58:40.999767+0000","last_unstale":"2024-03-24T02:59:30.970923+0000","last_undegraded":"2024-03-24T02:59:30.970923+0000","last_fullsized":"2024-03-24T02:59:30.970923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:31:44.927277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971379+0000","last_change":"2024-03-24T02:58:36.133118+0000","last_active":"2024-03-24T02:59:30.971379+0000","last_peered":"2024-03-24T02:59:30.971379+0000","last_clean":"2024-03-24T02:59:30.971379+0000","last_became_active":"2024-03-24T02:58:36.132747+0000","last_became_peered":"2024-03-24T02:58:36.132747+0000","last_unstale":"2024-03-24T02:59:30.971379+0000","last_undegraded":"2024-03-24T02:59:30.971379+0000","last_fullsized":"2024-03-24T02:59:30.971379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:11:15.665601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.996234+0000","last_change":"2024-03-24T02:58:36.985489+0000","last_active":"2024-03-24T02:59:59.996234+0000","last_peered":"2024-03-24T02:59:59.996234+0000","last_clean":"2024-03-24T02:59:59.996234+0000","last_became_active":"2024-03-24T02:58:36.984421+0000","last_became_peered":"2024-03-24T02:58:36.984421+0000","last_unstale":"2024-03-24T02:59:59.996234+0000","last_undegraded":"2024-03-24T02:59:59.996234+0000","last_fullsized":"2024-03-24T02:59:59.996234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:57:03.004888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:30.971389+0000","last_change":"2024-03-24T02:58:40.991456+0000","last_active":"2024-03-24T02:59:30.971389+0000","last_peered":"2024-03-24T02:59:30.971389+0000","last_clean":"2024-03-24T02:59:30.971389+0000","last_became_active":"2024-03-24T02:58:40.990973+0000","last_became_peered":"2024-03-24T02:58:40.990973+0000","last_unstale":"2024-03-24T02:59:30.971389+0000","last_undegraded":"2024-03-24T02:59:30.971389+0000","last_fullsized":"2024-03-24T02:59:30.971389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:39.971465+0000","last_clean_scrub_stamp":"2024-03-24T02:58:39.971465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:36:01.276988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:31.131981+0000","last_change":"2024-03-24T02:58:36.130967+0000","last_active":"2024-03-24T02:59:31.131981+0000","last_peered":"2024-03-24T02:59:31.131981+0000","last_clean":"2024-03-24T02:59:31.131981+0000","last_became_active":"2024-03-24T02:58:36.130657+0000","last_became_peered":"2024-03-24T02:58:36.130657+0000","last_unstale":"2024-03-24T02:59:31.131981+0000","last_undegraded":"2024-03-24T02:59:31.131981+0000","last_fullsized":"2024-03-24T02:59:31.131981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:35.108644+0000","last_clean_scrub_stamp":"2024-03-24T02:58:35.108644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:26:20.426922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.988460+0000","last_change":"2024-03-24T02:58:36.982297+0000","last_active":"2024-03-24T02:59:59.988460+0000","last_peered":"2024-03-24T02:59:59.988460+0000","last_clean":"2024-03-24T02:59:59.988460+0000","last_became_active":"2024-03-24T02:58:36.981792+0000","last_became_peered":"2024-03-24T02:58:36.981792+0000","last_unstale":"2024-03-24T02:59:59.988460+0000","last_undegraded":"2024-03-24T02:59:59.988460+0000","last_fullsized":"2024-03-24T02:59:59.988460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:03:07.528142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.996393+0000","last_change":"2024-03-24T02:58:36.979815+0000","last_active":"2024-03-24T02:59:59.996393+0000","last_peered":"2024-03-24T02:59:59.996393+0000","last_clean":"2024-03-24T02:59:59.996393+0000","last_became_active":"2024-03-24T02:58:36.979467+0000","last_became_peered":"2024-03-24T02:58:36.979467+0000","last_unstale":"2024-03-24T02:59:59.996393+0000","last_undegraded":"2024-03-24T02:59:59.996393+0000","last_fullsized":"2024-03-24T02:59:59.996393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:28.056630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.994790+0000","last_change":"2024-03-24T02:59:31.616266+0000","last_active":"2024-03-24T02:59:59.994790+0000","last_peered":"2024-03-24T02:59:59.994790+0000","last_clean":"2024-03-24T02:59:59.994790+0000","last_became_active":"2024-03-24T02:59:31.615872+0000","last_became_peered":"2024-03-24T02:59:31.615872+0000","last_unstale":"2024-03-24T02:59:59.994790+0000","last_undegraded":"2024-03-24T02:59:59.994790+0000","last_fullsized":"2024-03-24T02:59:59.994790+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:36:35.596949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.969229+0000","last_change":"2024-03-24T02:58:36.978207+0000","last_active":"2024-03-24T02:59:59.969229+0000","last_peered":"2024-03-24T02:59:59.969229+0000","last_clean":"2024-03-24T02:59:59.969229+0000","last_became_active":"2024-03-24T02:58:36.977760+0000","last_became_peered":"2024-03-24T02:58:36.977760+0000","last_unstale":"2024-03-24T02:59:59.969229+0000","last_undegraded":"2024-03-24T02:59:59.969229+0000","last_fullsized":"2024-03-24T02:59:59.969229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:43:51.603963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.994786+0000","last_change":"2024-03-24T02:59:31.132808+0000","last_active":"2024-03-24T02:59:59.994786+0000","last_peered":"2024-03-24T02:59:59.994786+0000","last_clean":"2024-03-24T02:59:59.994786+0000","last_became_active":"2024-03-24T02:59:31.132404+0000","last_became_peered":"2024-03-24T02:59:31.132404+0000","last_unstale":"2024-03-24T02:59:59.994786+0000","last_undegraded":"2024-03-24T02:59:59.994786+0000","last_fullsized":"2024-03-24T02:59:59.994786+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:55:07.143353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995763+0000","last_change":"2024-03-24T02:58:36.977261+0000","last_active":"2024-03-24T02:59:59.995763+0000","last_peered":"2024-03-24T02:59:59.995763+0000","last_clean":"2024-03-24T02:59:59.995763+0000","last_became_active":"2024-03-24T02:58:36.974644+0000","last_became_peered":"2024-03-24T02:58:36.974644+0000","last_unstale":"2024-03-24T02:59:59.995763+0000","last_undegraded":"2024-03-24T02:59:59.995763+0000","last_fullsized":"2024-03-24T02:59:59.995763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:34:50.968770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.984170+0000","last_change":"2024-03-24T02:58:36.981223+0000","last_active":"2024-03-24T02:59:59.984170+0000","last_peered":"2024-03-24T02:59:59.984170+0000","last_clean":"2024-03-24T02:59:59.984170+0000","last_became_active":"2024-03-24T02:58:36.979728+0000","last_became_peered":"2024-03-24T02:58:36.979728+0000","last_unstale":"2024-03-24T02:59:59.984170+0000","last_undegraded":"2024-03-24T02:59:59.984170+0000","last_fullsized":"2024-03-24T02:59:59.984170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:08:41.300442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.990102+0000","last_change":"2024-03-24T02:58:36.972385+0000","last_active":"2024-03-24T02:59:59.990102+0000","last_peered":"2024-03-24T02:59:59.990102+0000","last_clean":"2024-03-24T02:59:59.990102+0000","last_became_active":"2024-03-24T02:58:36.972018+0000","last_became_peered":"2024-03-24T02:58:36.972018+0000","last_unstale":"2024-03-24T02:59:59.990102+0000","last_undegraded":"2024-03-24T02:59:59.990102+0000","last_fullsized":"2024-03-24T02:59:59.990102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:02:22.529198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.987478+0000","last_change":"2024-03-24T02:59:31.132804+0000","last_active":"2024-03-24T02:59:59.987478+0000","last_peered":"2024-03-24T02:59:59.987478+0000","last_clean":"2024-03-24T02:59:59.987478+0000","last_became_active":"2024-03-24T02:59:31.132417+0000","last_became_peered":"2024-03-24T02:59:31.132417+0000","last_unstale":"2024-03-24T02:59:59.987478+0000","last_undegraded":"2024-03-24T02:59:59.987478+0000","last_fullsized":"2024-03-24T02:59:59.987478+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:17:03.492418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995489+0000","last_change":"2024-03-24T02:58:36.977256+0000","last_active":"2024-03-24T02:59:59.995489+0000","last_peered":"2024-03-24T02:59:59.995489+0000","last_clean":"2024-03-24T02:59:59.995489+0000","last_became_active":"2024-03-24T02:58:36.976888+0000","last_became_peered":"2024-03-24T02:58:36.976888+0000","last_unstale":"2024-03-24T02:59:59.995489+0000","last_undegraded":"2024-03-24T02:59:59.995489+0000","last_fullsized":"2024-03-24T02:59:59.995489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:18:19.449590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.987776+0000","last_change":"2024-03-24T02:58:36.981413+0000","last_active":"2024-03-24T02:59:59.987776+0000","last_peered":"2024-03-24T02:59:59.987776+0000","last_clean":"2024-03-24T02:59:59.987776+0000","last_became_active":"2024-03-24T02:58:36.980346+0000","last_became_peered":"2024-03-24T02:58:36.980346+0000","last_unstale":"2024-03-24T02:59:59.987776+0000","last_undegraded":"2024-03-24T02:59:59.987776+0000","last_fullsized":"2024-03-24T02:59:59.987776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:17:57.332228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T03:00:00.000008+0000","last_change":"2024-03-24T02:58:36.984264+0000","last_active":"2024-03-24T03:00:00.000008+0000","last_peered":"2024-03-24T03:00:00.000008+0000","last_clean":"2024-03-24T03:00:00.000008+0000","last_became_active":"2024-03-24T02:58:36.983905+0000","last_became_peered":"2024-03-24T02:58:36.983905+0000","last_unstale":"2024-03-24T03:00:00.000008+0000","last_undegraded":"2024-03-24T03:00:00.000008+0000","last_fullsized":"2024-03-24T03:00:00.000008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:25:17.872320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.995968+0000","last_change":"2024-03-24T02:58:36.975939+0000","last_active":"2024-03-24T02:59:59.995968+0000","last_peered":"2024-03-24T02:59:59.995968+0000","last_clean":"2024-03-24T02:59:59.995968+0000","last_became_active":"2024-03-24T02:58:36.975477+0000","last_became_peered":"2024-03-24T02:58:36.975477+0000","last_unstale":"2024-03-24T02:59:59.995968+0000","last_undegraded":"2024-03-24T02:59:59.995968+0000","last_fullsized":"2024-03-24T02:59:59.995968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:46:43.603391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-24T02:59:59.991554+0000","last_change":"2024-03-24T02:58:36.980674+0000","last_active":"2024-03-24T02:59:59.991554+0000","last_peered":"2024-03-24T02:59:59.991554+0000","last_clean":"2024-03-24T02:59:59.991554+0000","last_became_active":"2024-03-24T02:58:36.980278+0000","last_became_peered":"2024-03-24T02:58:36.980278+0000","last_unstale":"2024-03-24T02:59:59.991554+0000","last_undegraded":"2024-03-24T02:59:59.991554+0000","last_fullsized":"2024-03-24T02:59:59.991554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T02:58:36.109979+0000","last_clean_scrub_stamp":"2024-03-24T02:58:36.109979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:50:01.719362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74479,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":772,"num_read_kb":471,"num_write":249,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":223437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":501823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1155632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":505317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":501599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1098237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":496725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":564748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":498713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":565500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":496433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":501331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1658880,"data_stored":1093287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-24T03:00:15.684 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-24T03:00:15.684 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-24T03:00:15.684 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-24T03:00:15.684 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-24T03:00:15.930 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:00:15.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:15 smithi017 ceph-mon[31926]: pgmap v343: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-24T03:00:15.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:15 smithi017 ceph-mon[31926]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T03:00:16.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:15 smithi188 ceph-mon[36206]: pgmap v343: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-24T03:00:16.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:15 smithi188 ceph-mon[36206]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T03:00:16.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:15 smithi019 ceph-mon[35944]: pgmap v343: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-24T03:00:16.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:15 smithi019 ceph-mon[35944]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T03:00:17.805 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:17 smithi017 ceph-mon[31926]: pgmap v344: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.817+0000 7f469ecbc700 1 -- 172.21.15.17:0/1763340817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46981070f0 msgr2=0x7f46981074d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.817+0000 7f469ecbc700 1 --2- 172.21.15.17:0/1763340817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46981070f0 0x7f46981074d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f468c009990 tx=0x7f468c009ca0 comp rx=0 tx=0).stop 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.817+0000 7f469ecbc700 1 -- 172.21.15.17:0/1763340817 shutdown_connections 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.817+0000 7f469ecbc700 1 --2- 172.21.15.17:0/1763340817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698101a90 0x7f469810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.817+0000 7f469ecbc700 1 --2- 172.21.15.17:0/1763340817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46981070f0 0x7f46981074d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.817+0000 7f469ecbc700 1 --2- 172.21.15.17:0/1763340817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46981010f0 0x7f4698101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.817+0000 7f469ecbc700 1 -- 172.21.15.17:0/1763340817 >> 172.21.15.17:0/1763340817 conn(0x7f46980770d0 msgr2=0x7f46980774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.817+0000 7f469ecbc700 1 -- 172.21.15.17:0/1763340817 shutdown_connections 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.817+0000 7f469ecbc700 1 -- 172.21.15.17:0/1763340817 wait complete. 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f469ecbc700 1 Processor -- start 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f469ecbc700 1 -- start start 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f469ecbc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46981010f0 0x7f469810e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f469ecbc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4698101a90 0x7f469810ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f469ecbc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46981070f0 0x7f469810f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f469ecbc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46981124b0 con 0x7f46981010f0 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f469ecbc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4698112330 con 0x7f46981070f0 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f469ecbc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4698112630 con 0x7f4698101a90 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f469ca58700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46981010f0 0x7f469810e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f469ca58700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46981010f0 0x7f469810e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53060/0 (socket says 172.21.15.17:53060) 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f469ca58700 1 -- 172.21.15.17:0/1316160320 learned_addr learned my addr 172.21.15.17:0/1316160320 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f4697fff700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4698101a90 0x7f469810ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.818+0000 7f469d259700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46981070f0 0x7f469810f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.819+0000 7f469ca58700 1 -- 172.21.15.17:0/1316160320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4698101a90 msgr2=0x7f469810ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.819+0000 7f469ca58700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4698101a90 0x7f469810ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.819+0000 7f469ca58700 1 -- 172.21.15.17:0/1316160320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46981070f0 msgr2=0x7f469810f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.819+0000 7f469ca58700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46981070f0 0x7f469810f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.819+0000 7f469ca58700 1 -- 172.21.15.17:0/1316160320 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46981a6ef0 con 0x7f46981010f0 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.819+0000 7f469ca58700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46981010f0 0x7f469810e9c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f468c000c00 tx=0x7f468c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.819+0000 7f4695ffb700 1 -- 172.21.15.17:0/1316160320 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c027070 con 0x7f46981010f0 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.819+0000 7f4695ffb700 1 -- 172.21.15.17:0/1316160320 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f468c003d70 con 0x7f46981010f0 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.819+0000 7f469ecbc700 1 -- 172.21.15.17:0/1316160320 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46981a7180 con 0x7f46981010f0 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.819+0000 7f4695ffb700 1 -- 172.21.15.17:0/1316160320 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c00f460 con 0x7f46981010f0 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.819+0000 7f469ecbc700 1 -- 172.21.15.17:0/1316160320 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46981a75d0 con 0x7f46981010f0 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.820+0000 7f469ecbc700 1 -- 172.21.15.17:0/1316160320 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4698068e50 con 0x7f46981010f0 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.821+0000 7f4695ffb700 1 -- 172.21.15.17:0/1316160320 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f468c02a070 con 0x7f46981010f0 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.822+0000 7f4695ffb700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f46880a2890 0x7f46880a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.822+0000 7f4695ffb700 1 -- 172.21.15.17:0/1316160320 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f468c0cff10 con 0x7f46981010f0 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.822+0000 7f4697fff700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f46880a2890 0x7f46880a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:17.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.823+0000 7f4697fff700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f46880a2890 0x7f46880a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4698077790 tx=0x7f4684009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:17.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:17.826+0000 7f4695ffb700 1 -- 172.21.15.17:0/1316160320 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f468c013070 con 0x7f46981010f0 2024-03-24T03:00:18.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:17 smithi188 ceph-mon[36206]: pgmap v344: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-24T03:00:18.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:17 smithi019 ceph-mon[35944]: pgmap v344: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-24T03:00:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.079+0000 7f469ecbc700 1 -- 172.21.15.17:0/1316160320 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f469802da30 con 0x7f46981010f0 2024-03-24T03:00:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.079+0000 7f4695ffb700 1 -- 172.21.15.17:0/1316160320 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f468c09ccb0 con 0x7f46981010f0 2024-03-24T03:00:18.078 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:18.079 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-24T03:00:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.082+0000 7f469ecbc700 1 -- 172.21.15.17:0/1316160320 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f46880a2890 msgr2=0x7f46880a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.082+0000 7f469ecbc700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f46880a2890 0x7f46880a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4698077790 tx=0x7f4684009260 comp rx=0 tx=0).stop 2024-03-24T03:00:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.082+0000 7f469ecbc700 1 -- 172.21.15.17:0/1316160320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46981010f0 msgr2=0x7f469810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.082+0000 7f469ecbc700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46981010f0 0x7f469810e9c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f468c000c00 tx=0x7f468c00bab0 comp rx=0 tx=0).stop 2024-03-24T03:00:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.082+0000 7f469ecbc700 1 -- 172.21.15.17:0/1316160320 shutdown_connections 2024-03-24T03:00:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.082+0000 7f469ecbc700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f46880a2890 0x7f46880a4d50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.082+0000 7f469ecbc700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46981070f0 0x7f469810f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.082+0000 7f469ecbc700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46981010f0 0x7f469810e9c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.082+0000 7f469ecbc700 1 --2- 172.21.15.17:0/1316160320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4698101a90 0x7f469810ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:18.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.082+0000 7f469ecbc700 1 -- 172.21.15.17:0/1316160320 >> 172.21.15.17:0/1316160320 conn(0x7f46980770d0 msgr2=0x7f46980ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.082+0000 7f469ecbc700 1 -- 172.21.15.17:0/1316160320 shutdown_connections 2024-03-24T03:00:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:18.082+0000 7f469ecbc700 1 -- 172.21.15.17:0/1316160320 wait complete. 2024-03-24T03:00:18.891 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-24T03:00:18.891 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T03:00:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.002+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3540260066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ecc103270 msgr2=0x7f3ecc109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.002+0000 7f3ed3dbc640 1 --2- 172.21.15.17:0/3540260066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ecc103270 0x7f3ecc109b00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009ca0 tx=0x7f3ec402c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.002+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3540260066 shutdown_connections 2024-03-24T03:00:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.002+0000 7f3ed3dbc640 1 --2- 172.21.15.17:0/3540260066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ecc103270 0x7f3ecc109b00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.002+0000 7f3ed3dbc640 1 --2- 172.21.15.17:0/3540260066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc1028b0 0x7f3ecc102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.002+0000 7f3ed3dbc640 1 --2- 172.21.15.17:0/3540260066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ecc1016c0 0x7f3ecc101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.002+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3540260066 >> 172.21.15.17:0/3540260066 conn(0x7f3ecc0fcc50 msgr2=0x7f3ecc0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.002+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3540260066 shutdown_connections 2024-03-24T03:00:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.002+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3540260066 wait complete. 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed3dbc640 1 Processor -- start 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed3dbc640 1 -- start start 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed3dbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ecc1016c0 0x7f3ecc1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed3dbc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ecc1028b0 0x7f3ecc197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed3dbc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc103270 0x7f3ecc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed3dbc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc10d320 con 0x7f3ecc1016c0 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed3dbc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc10d1a0 con 0x7f3ecc103270 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed3dbc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc10d4a0 con 0x7f3ecc1028b0 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed1b58640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ecc1016c0 0x7f3ecc1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed1b58640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ecc1016c0 0x7f3ecc1979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53070/0 (socket says 172.21.15.17:53070) 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed1b58640 1 -- 172.21.15.17:0/3248028344 learned_addr learned my addr 172.21.15.17:0/3248028344 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed1357640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ecc1028b0 0x7f3ecc197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.003+0000 7f3ed2359640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc103270 0x7f3ecc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.004+0000 7f3ed1b58640 1 -- 172.21.15.17:0/3248028344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ecc1028b0 msgr2=0x7f3ecc197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.004+0000 7f3ed1b58640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ecc1028b0 0x7f3ecc197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.004+0000 7f3ed1b58640 1 -- 172.21.15.17:0/3248028344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc103270 msgr2=0x7f3ecc19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.004+0000 7f3ed1b58640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc103270 0x7f3ecc19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.004+0000 7f3ed1b58640 1 -- 172.21.15.17:0/3248028344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc19f5f0 con 0x7f3ecc1016c0 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.004+0000 7f3ed1b58640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ecc1016c0 0x7f3ecc1979d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800e9c0 tx=0x7f3ec800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.004+0000 7f3ec2ffd640 1 -- 172.21.15.17:0/3248028344 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800cd80 con 0x7f3ecc1016c0 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.004+0000 7f3ec2ffd640 1 -- 172.21.15.17:0/3248028344 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ec80035f0 con 0x7f3ecc1016c0 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.004+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3248028344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ecc19f8e0 con 0x7f3ecc1016c0 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.004+0000 7f3ec2ffd640 1 -- 172.21.15.17:0/3248028344 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8010690 con 0x7f3ecc1016c0 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.004+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3248028344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc19fe30 con 0x7f3ecc1016c0 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.005+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3248028344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ecc0688e0 con 0x7f3ecc1016c0 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.006+0000 7f3ec2ffd640 1 -- 172.21.15.17:0/3248028344 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3ec80037a0 con 0x7f3ecc1016c0 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.007+0000 7f3ec2ffd640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3eb80a2800 0x7f3eb80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.007+0000 7f3ed1357640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3eb80a2800 0x7f3eb80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.007+0000 7f3ec2ffd640 1 -- 172.21.15.17:0/3248028344 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f3ec80cf140 con 0x7f3ecc1016c0 2024-03-24T03:00:19.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.007+0000 7f3ed1357640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3eb80a2800 0x7f3eb80a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc198ee0 tx=0x7f3ebc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:19.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.011+0000 7f3ec2ffd640 1 -- 172.21.15.17:0/3248028344 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ec809beb0 con 0x7f3ecc1016c0 2024-03-24T03:00:19.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:18 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1316160320' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T03:00:19.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:18 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1316160320' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T03:00:19.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.200+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3248028344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ecc0622a0 con 0x7f3ecc1016c0 2024-03-24T03:00:19.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.202+0000 7f3ec2ffd640 1 -- 172.21.15.17:0/3248028344 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24514 (secure 0 0 0) 0x7f3ec809b7b0 con 0x7f3ecc1016c0 2024-03-24T03:00:19.202 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:19.202 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":91,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:59:30.964668+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T03:00:19.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.206+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3248028344 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3eb80a2800 msgr2=0x7f3eb80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.206+0000 7f3ed3dbc640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3eb80a2800 0x7f3eb80a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc198ee0 tx=0x7f3ebc005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:19.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.206+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3248028344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ecc1016c0 msgr2=0x7f3ecc1979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.206+0000 7f3ed3dbc640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ecc1016c0 0x7f3ecc1979d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800e9c0 tx=0x7f3ec800ee80 comp rx=0 tx=0).stop 2024-03-24T03:00:19.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.206+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3248028344 shutdown_connections 2024-03-24T03:00:19.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.206+0000 7f3ed3dbc640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3eb80a2800 0x7f3eb80a4cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.206+0000 7f3ed3dbc640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc103270 0x7f3ecc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.206+0000 7f3ed3dbc640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ecc1028b0 0x7f3ecc197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.206+0000 7f3ed3dbc640 1 --2- 172.21.15.17:0/3248028344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ecc1016c0 0x7f3ecc1979d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.206+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3248028344 >> 172.21.15.17:0/3248028344 conn(0x7f3ecc0fcc50 msgr2=0x7f3ecc105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:19.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.206+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3248028344 shutdown_connections 2024-03-24T03:00:19.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.206+0000 7f3ed3dbc640 1 -- 172.21.15.17:0/3248028344 wait complete. 2024-03-24T03:00:19.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:18 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1316160320' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T03:00:19.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T02:53:54.207139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-24T02:58:34.785192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-24T02:58:35.679772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-24T02:58:39.221772+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-24T03:00:19.221 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T03:00:19.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.328+0000 7ff356aa1640 1 -- 172.21.15.17:0/2433013417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3500ff970 msgr2=0x7ff3500ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.328+0000 7ff356aa1640 1 --2- 172.21.15.17:0/2433013417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3500ff970 0x7ff3500ffd90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff344009930 tx=0x7ff34402bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.328+0000 7ff356aa1640 1 -- 172.21.15.17:0/2433013417 shutdown_connections 2024-03-24T03:00:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.328+0000 7ff356aa1640 1 --2- 172.21.15.17:0/2433013417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff350103300 0x7ff350109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.328+0000 7ff356aa1640 1 --2- 172.21.15.17:0/2433013417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff350102940 0x7ff350102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.328+0000 7ff356aa1640 1 --2- 172.21.15.17:0/2433013417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3500ff970 0x7ff3500ffd90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.328+0000 7ff356aa1640 1 -- 172.21.15.17:0/2433013417 >> 172.21.15.17:0/2433013417 conn(0x7ff3500fb2c0 msgr2=0x7ff3500fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.328+0000 7ff356aa1640 1 -- 172.21.15.17:0/2433013417 shutdown_connections 2024-03-24T03:00:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.328+0000 7ff356aa1640 1 -- 172.21.15.17:0/2433013417 wait complete. 2024-03-24T03:00:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.328+0000 7ff356aa1640 1 Processor -- start 2024-03-24T03:00:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.328+0000 7ff356aa1640 1 -- start start 2024-03-24T03:00:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff356aa1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3500ff970 0x7ff3501022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff356aa1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff350102940 0x7ff3501008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff356aa1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff350103300 0x7ff350100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff356aa1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35010d370 con 0x7ff350102940 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff356aa1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35010d1f0 con 0x7ff350103300 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff356aa1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35010d4f0 con 0x7ff3500ff970 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff350102940 0x7ff3501008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff350102940 0x7ff3501008f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53100/0 (socket says 172.21.15.17:53100) 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff34ffff640 1 -- 172.21.15.17:0/694693362 learned_addr learned my addr 172.21.15.17:0/694693362 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff35483d640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3500ff970 0x7ff3501022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff35503e640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff350103300 0x7ff350100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff34ffff640 1 -- 172.21.15.17:0/694693362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3500ff970 msgr2=0x7ff3501022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff34ffff640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3500ff970 0x7ff3501022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff34ffff640 1 -- 172.21.15.17:0/694693362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff350103300 msgr2=0x7ff350100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff34ffff640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff350103300 0x7ff350100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff34ffff640 1 -- 172.21.15.17:0/694693362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3501016c0 con 0x7ff350102940 2024-03-24T03:00:19.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff34ffff640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff350102940 0x7ff3501008f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00e9f0 tx=0x7ff33c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:19.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff34dffb640 1 -- 172.21.15.17:0/694693362 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00cd80 con 0x7ff350102940 2024-03-24T03:00:19.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff34dffb640 1 -- 172.21.15.17:0/694693362 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff33c010910 con 0x7ff350102940 2024-03-24T03:00:19.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff34dffb640 1 -- 172.21.15.17:0/694693362 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c018990 con 0x7ff350102940 2024-03-24T03:00:19.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.329+0000 7ff356aa1640 1 -- 172.21.15.17:0/694693362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3501a1b30 con 0x7ff350102940 2024-03-24T03:00:19.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.330+0000 7ff356aa1640 1 -- 172.21.15.17:0/694693362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3501a2080 con 0x7ff350102940 2024-03-24T03:00:19.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.330+0000 7ff356aa1640 1 -- 172.21.15.17:0/694693362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff350108af0 con 0x7ff350102940 2024-03-24T03:00:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.331+0000 7ff34dffb640 1 -- 172.21.15.17:0/694693362 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff33c010ac0 con 0x7ff350102940 2024-03-24T03:00:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.332+0000 7ff34dffb640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff3400a2800 0x7ff3400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.332+0000 7ff35483d640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff3400a2800 0x7ff3400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.332+0000 7ff34dffb640 1 -- 172.21.15.17:0/694693362 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7ff33c0cf0b0 con 0x7ff350102940 2024-03-24T03:00:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.332+0000 7ff35483d640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff3400a2800 0x7ff3400a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff344009230 tx=0x7ff34402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:19.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.336+0000 7ff34dffb640 1 -- 172.21.15.17:0/694693362 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff33c09b7d0 con 0x7ff350102940 2024-03-24T03:00:19.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.517+0000 7ff356aa1640 1 -- 172.21.15.17:0/694693362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff350062340 con 0x7ff350102940 2024-03-24T03:00:19.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.518+0000 7ff34dffb640 1 -- 172.21.15.17:0/694693362 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff33c09f870 con 0x7ff350102940 2024-03-24T03:00:19.517 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-24T03:00:19.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.520+0000 7ff356aa1640 1 -- 172.21.15.17:0/694693362 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff3400a2800 msgr2=0x7ff3400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.520+0000 7ff356aa1640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff3400a2800 0x7ff3400a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff344009230 tx=0x7ff34402df90 comp rx=0 tx=0).stop 2024-03-24T03:00:19.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.520+0000 7ff356aa1640 1 -- 172.21.15.17:0/694693362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff350102940 msgr2=0x7ff3501008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.520+0000 7ff356aa1640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff350102940 0x7ff3501008f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00e9f0 tx=0x7ff33c00eeb0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.520+0000 7ff356aa1640 1 -- 172.21.15.17:0/694693362 shutdown_connections 2024-03-24T03:00:19.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.520+0000 7ff356aa1640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff3400a2800 0x7ff3400a4cc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.520+0000 7ff356aa1640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff350103300 0x7ff350100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.520+0000 7ff356aa1640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff350102940 0x7ff3501008f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.520+0000 7ff356aa1640 1 --2- 172.21.15.17:0/694693362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3500ff970 0x7ff3501022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.520+0000 7ff356aa1640 1 -- 172.21.15.17:0/694693362 >> 172.21.15.17:0/694693362 conn(0x7ff3500fb2c0 msgr2=0x7ff350105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:19.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.520+0000 7ff356aa1640 1 -- 172.21.15.17:0/694693362 shutdown_connections 2024-03-24T03:00:19.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.520+0000 7ff356aa1640 1 -- 172.21.15.17:0/694693362 wait complete. 2024-03-24T03:00:19.531 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-24T03:00:19.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.637+0000 7fb0cf909640 1 -- 172.21.15.17:0/578481904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c8102160 msgr2=0x7fb0c81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.637+0000 7fb0cf909640 1 --2- 172.21.15.17:0/578481904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c8102160 0x7fb0c81089f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009ca0 tx=0x7fb0c002c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.637+0000 7fb0cf909640 1 -- 172.21.15.17:0/578481904 shutdown_connections 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.637+0000 7fb0cf909640 1 --2- 172.21.15.17:0/578481904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c8102160 0x7fb0c81089f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.637+0000 7fb0cf909640 1 --2- 172.21.15.17:0/578481904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0c81017a0 0x7fb0c8101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.637+0000 7fb0cf909640 1 --2- 172.21.15.17:0/578481904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c81005b0 0x7fb0c81009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.637+0000 7fb0cf909640 1 -- 172.21.15.17:0/578481904 >> 172.21.15.17:0/578481904 conn(0x7fb0c80fbb40 msgr2=0x7fb0c80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.637+0000 7fb0cf909640 1 -- 172.21.15.17:0/578481904 shutdown_connections 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.637+0000 7fb0cf909640 1 -- 172.21.15.17:0/578481904 wait complete. 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0cf909640 1 Processor -- start 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0cf909640 1 -- start start 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0cf909640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c81005b0 0x7fb0c8104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0cf909640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c81017a0 0x7fb0c8102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0cf909640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0c8102160 0x7fb0c81030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0cf909640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c810c210 con 0x7fb0c81017a0 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0cf909640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c810c090 con 0x7fb0c8102160 2024-03-24T03:00:19.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0cf909640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c810c390 con 0x7fb0c81005b0 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0ccea4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c81017a0 0x7fb0c8102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0cd6a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c81005b0 0x7fb0c8104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0cdea6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0c8102160 0x7fb0c81030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0ccea4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c81017a0 0x7fb0c8102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53130/0 (socket says 172.21.15.17:53130) 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0cd6a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c81005b0 0x7fb0c8104540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:59894/0 (socket says 172.21.15.17:59894) 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0ccea4640 1 -- 172.21.15.17:0/3603240631 learned_addr learned my addr 172.21.15.17:0/3603240631 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0ccea4640 1 -- 172.21.15.17:0/3603240631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c81005b0 msgr2=0x7fb0c8104540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0ccea4640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c81005b0 0x7fb0c8104540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0ccea4640 1 -- 172.21.15.17:0/3603240631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0c8102160 msgr2=0x7fb0c81030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0ccea4640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0c8102160 0x7fb0c81030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0ccea4640 1 -- 172.21.15.17:0/3603240631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c8103950 con 0x7fb0c81017a0 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0cdea6640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0c8102160 0x7fb0c81030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.638+0000 7fb0ccea4640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c81017a0 0x7fb0c8102b80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800e9f0 tx=0x7fb0b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:19.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.639+0000 7fb0be7fc640 1 -- 172.21.15.17:0/3603240631 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800cd80 con 0x7fb0c81017a0 2024-03-24T03:00:19.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.639+0000 7fb0be7fc640 1 -- 172.21.15.17:0/3603240631 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0b8010910 con 0x7fb0c81017a0 2024-03-24T03:00:19.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.639+0000 7fb0cf909640 1 -- 172.21.15.17:0/3603240631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c81a1ac0 con 0x7fb0c81017a0 2024-03-24T03:00:19.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.639+0000 7fb0be7fc640 1 -- 172.21.15.17:0/3603240631 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8018990 con 0x7fb0c81017a0 2024-03-24T03:00:19.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.639+0000 7fb0cf909640 1 -- 172.21.15.17:0/3603240631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c81a2010 con 0x7fb0c81017a0 2024-03-24T03:00:19.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.640+0000 7fb0cf909640 1 -- 172.21.15.17:0/3603240631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c81a21e0 con 0x7fb0c81017a0 2024-03-24T03:00:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.641+0000 7fb0be7fc640 1 -- 172.21.15.17:0/3603240631 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb0b8010ac0 con 0x7fb0c81017a0 2024-03-24T03:00:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.641+0000 7fb0be7fc640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb0b40a2800 0x7fb0b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.642+0000 7fb0cd6a5640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb0b40a2800 0x7fb0b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.642+0000 7fb0be7fc640 1 -- 172.21.15.17:0/3603240631 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fb0b80cf510 con 0x7fb0c81017a0 2024-03-24T03:00:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.642+0000 7fb0cd6a5640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb0b40a2800 0x7fb0b40a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40095f0 tx=0x7fb0c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:19.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.645+0000 7fb0be7fc640 1 -- 172.21.15.17:0/3603240631 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb0b809bbd0 con 0x7fb0c81017a0 2024-03-24T03:00:19.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.828+0000 7fb0cf909640 1 -- 172.21.15.17:0/3603240631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb0c81a2570 con 0x7fb0c81017a0 2024-03-24T03:00:19.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.828+0000 7fb0be7fc640 1 -- 172.21.15.17:0/3603240631 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb0b809fc70 con 0x7fb0c81017a0 2024-03-24T03:00:19.828 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T03:00:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.830+0000 7fb0cf909640 1 -- 172.21.15.17:0/3603240631 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb0b40a2800 msgr2=0x7fb0b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.830+0000 7fb0cf909640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb0b40a2800 0x7fb0b40a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40095f0 tx=0x7fb0c4005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.830+0000 7fb0cf909640 1 -- 172.21.15.17:0/3603240631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c81017a0 msgr2=0x7fb0c8102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.830+0000 7fb0cf909640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c81017a0 0x7fb0c8102b80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800e9f0 tx=0x7fb0b800eeb0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.830+0000 7fb0cf909640 1 -- 172.21.15.17:0/3603240631 shutdown_connections 2024-03-24T03:00:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.830+0000 7fb0cf909640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb0b40a2800 0x7fb0b40a4cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.830+0000 7fb0cf909640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0c8102160 0x7fb0c81030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.830+0000 7fb0cf909640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c81017a0 0x7fb0c8102b80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.830+0000 7fb0cf909640 1 --2- 172.21.15.17:0/3603240631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c81005b0 0x7fb0c8104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.830+0000 7fb0cf909640 1 -- 172.21.15.17:0/3603240631 >> 172.21.15.17:0/3603240631 conn(0x7fb0c80fbb40 msgr2=0x7fb0c80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.830+0000 7fb0cf909640 1 -- 172.21.15.17:0/3603240631 shutdown_connections 2024-03-24T03:00:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.830+0000 7fb0cf909640 1 -- 172.21.15.17:0/3603240631 wait complete. 2024-03-24T03:00:19.841 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-24T03:00:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.946+0000 7f1c67190640 1 -- 172.21.15.17:0/1690939144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c601073a0 msgr2=0x7f1c601097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.946+0000 7f1c67190640 1 --2- 172.21.15.17:0/1690939144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c601073a0 0x7f1c601097f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009ca0 tx=0x7f1c5802c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.946+0000 7f1c67190640 1 -- 172.21.15.17:0/1690939144 shutdown_connections 2024-03-24T03:00:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.946+0000 7f1c67190640 1 --2- 172.21.15.17:0/1690939144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c601073a0 0x7f1c601097f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.946+0000 7f1c67190640 1 --2- 172.21.15.17:0/1690939144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c60104a70 0x7f1c60106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.946+0000 7f1c67190640 1 --2- 172.21.15.17:0/1690939144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c60102140 0x7f1c60104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.946+0000 7f1c67190640 1 -- 172.21.15.17:0/1690939144 >> 172.21.15.17:0/1690939144 conn(0x7f1c600fbb50 msgr2=0x7f1c600fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.946+0000 7f1c67190640 1 -- 172.21.15.17:0/1690939144 shutdown_connections 2024-03-24T03:00:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.946+0000 7f1c67190640 1 -- 172.21.15.17:0/1690939144 wait complete. 2024-03-24T03:00:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c67190640 1 Processor -- start 2024-03-24T03:00:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c67190640 1 -- start start 2024-03-24T03:00:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c67190640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c60102140 0x7f1c60197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c67190640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c60104a70 0x7f1c60197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c67190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c601073a0 0x7f1c6019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c67190640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6010d360 con 0x7f1c60102140 2024-03-24T03:00:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c67190640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6010d1e0 con 0x7f1c601073a0 2024-03-24T03:00:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c67190640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6010d4e0 con 0x7f1c60104a70 2024-03-24T03:00:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c64f2c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c60102140 0x7f1c60197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c64f2c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c60102140 0x7f1c60197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53148/0 (socket says 172.21.15.17:53148) 2024-03-24T03:00:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c64f2c640 1 -- 172.21.15.17:0/3842415167 learned_addr learned my addr 172.21.15.17:0/3842415167 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c5ffff640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c60104a70 0x7f1c60197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c6572d640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c601073a0 0x7f1c6019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c64f2c640 1 -- 172.21.15.17:0/3842415167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c60104a70 msgr2=0x7f1c60197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c64f2c640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c60104a70 0x7f1c60197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c64f2c640 1 -- 172.21.15.17:0/3842415167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c601073a0 msgr2=0x7f1c6019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c64f2c640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c601073a0 0x7f1c6019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c64f2c640 1 -- 172.21.15.17:0/3842415167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6019f5a0 con 0x7f1c60102140 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.947+0000 7f1c64f2c640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c60102140 0x7f1c60197980 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400e9f0 tx=0x7f1c5400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.948+0000 7f1c5dffb640 1 -- 172.21.15.17:0/3842415167 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400cd80 con 0x7f1c60102140 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.948+0000 7f1c5dffb640 1 -- 172.21.15.17:0/3842415167 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c54005500 con 0x7f1c60102140 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.948+0000 7f1c67190640 1 -- 172.21.15.17:0/3842415167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6019f890 con 0x7f1c60102140 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.948+0000 7f1c5dffb640 1 -- 172.21.15.17:0/3842415167 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54010690 con 0x7f1c60102140 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.948+0000 7f1c67190640 1 -- 172.21.15.17:0/3842415167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6019fd90 con 0x7f1c60102140 2024-03-24T03:00:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.949+0000 7f1c67190640 1 -- 172.21.15.17:0/3842415167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c600fd640 con 0x7f1c60102140 2024-03-24T03:00:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.950+0000 7f1c5dffb640 1 -- 172.21.15.17:0/3842415167 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1c540046c0 con 0x7f1c60102140 2024-03-24T03:00:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.950+0000 7f1c5dffb640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1c500a2800 0x7f1c500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.951+0000 7f1c5ffff640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1c500a2800 0x7f1c500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.951+0000 7f1c5dffb640 1 -- 172.21.15.17:0/3842415167 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f1c540cf180 con 0x7f1c60102140 2024-03-24T03:00:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.951+0000 7f1c5ffff640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1c500a2800 0x7f1c500a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1c60198e90 tx=0x7f1c4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:19.955+0000 7f1c5dffb640 1 -- 172.21.15.17:0/3842415167 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1c5409bef0 con 0x7f1c60102140 2024-03-24T03:00:20.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:19 smithi188 ceph-mon[36206]: pgmap v345: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T03:00:20.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:19 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3248028344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:20.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:19 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/694693362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:00:20.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:19 smithi019 ceph-mon[35944]: pgmap v345: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T03:00:20.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:19 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3248028344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:20.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:19 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/694693362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:00:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.140+0000 7f1c67190640 1 -- 172.21.15.17:0/3842415167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c6002d070 con 0x7f1c60102140 2024-03-24T03:00:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.140+0000 7f1c5dffb640 1 -- 172.21.15.17:0/3842415167 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c5409b7f0 con 0x7f1c60102140 2024-03-24T03:00:20.140 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-24T03:00:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.142+0000 7f1c67190640 1 -- 172.21.15.17:0/3842415167 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1c500a2800 msgr2=0x7f1c500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.142+0000 7f1c67190640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1c500a2800 0x7f1c500a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1c60198e90 tx=0x7f1c4c005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.142+0000 7f1c67190640 1 -- 172.21.15.17:0/3842415167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c60102140 msgr2=0x7f1c60197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.142+0000 7f1c67190640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c60102140 0x7f1c60197980 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400e9f0 tx=0x7f1c5400edf0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.142+0000 7f1c67190640 1 -- 172.21.15.17:0/3842415167 shutdown_connections 2024-03-24T03:00:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.142+0000 7f1c67190640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1c500a2800 0x7f1c500a4cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.142+0000 7f1c67190640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c601073a0 0x7f1c6019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.142+0000 7f1c67190640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c60104a70 0x7f1c60197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.142+0000 7f1c67190640 1 --2- 172.21.15.17:0/3842415167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c60102140 0x7f1c60197980 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.142+0000 7f1c67190640 1 -- 172.21.15.17:0/3842415167 >> 172.21.15.17:0/3842415167 conn(0x7f1c600fbb50 msgr2=0x7f1c60102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:20.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.142+0000 7f1c67190640 1 -- 172.21.15.17:0/3842415167 shutdown_connections 2024-03-24T03:00:20.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.142+0000 7f1c67190640 1 -- 172.21.15.17:0/3842415167 wait complete. 2024-03-24T03:00:20.152 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:19 smithi017 ceph-mon[31926]: pgmap v345: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T03:00:20.152 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:19 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3248028344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:20.152 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:19 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/694693362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:00:20.153 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-24T03:00:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 -- 172.21.15.17:0/1309167436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcc4107380 msgr2=0x7fbcc41097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 --2- 172.21.15.17:0/1309167436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcc4107380 0x7fbcc41097d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009ca0 tx=0x7fbcbc02c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 -- 172.21.15.17:0/1309167436 shutdown_connections 2024-03-24T03:00:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 --2- 172.21.15.17:0/1309167436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcc4107380 0x7fbcc41097d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 --2- 172.21.15.17:0/1309167436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc4104a50 0x7fbcc4106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 --2- 172.21.15.17:0/1309167436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4102120 0x7fbcc4104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 -- 172.21.15.17:0/1309167436 >> 172.21.15.17:0/1309167436 conn(0x7fbcc40fbb30 msgr2=0x7fbcc40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 -- 172.21.15.17:0/1309167436 shutdown_connections 2024-03-24T03:00:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 -- 172.21.15.17:0/1309167436 wait complete. 2024-03-24T03:00:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 Processor -- start 2024-03-24T03:00:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 -- start start 2024-03-24T03:00:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcc4102120 0x7fbcc41979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc4104a50 0x7fbcc4197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4107380 0x7fbcc419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc410d330 con 0x7fbcc4102120 2024-03-24T03:00:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc410d1b0 con 0x7fbcc4107380 2024-03-24T03:00:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.259+0000 7fbccc67e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc410d4b0 con 0x7fbcc4104a50 2024-03-24T03:00:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcca41a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcc4102120 0x7fbcc41979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcca41a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcc4102120 0x7fbcc41979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53164/0 (socket says 172.21.15.17:53164) 2024-03-24T03:00:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcca41a640 1 -- 172.21.15.17:0/1846879917 learned_addr learned my addr 172.21.15.17:0/1846879917 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcc9c19640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc4104a50 0x7fbcc4197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbccac1b640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4107380 0x7fbcc419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcca41a640 1 -- 172.21.15.17:0/1846879917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc4104a50 msgr2=0x7fbcc4197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcca41a640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc4104a50 0x7fbcc4197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcca41a640 1 -- 172.21.15.17:0/1846879917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4107380 msgr2=0x7fbcc419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcca41a640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4107380 0x7fbcc419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcca41a640 1 -- 172.21.15.17:0/1846879917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc419f600 con 0x7fbcc4102120 2024-03-24T03:00:20.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcca41a640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcc4102120 0x7fbcc41979e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000e9f0 tx=0x7fbcc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:20.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcbb7fe640 1 -- 172.21.15.17:0/1846879917 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000cd80 con 0x7fbcc4102120 2024-03-24T03:00:20.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcbb7fe640 1 -- 172.21.15.17:0/1846879917 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcc0010910 con 0x7fbcc4102120 2024-03-24T03:00:20.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbccc67e640 1 -- 172.21.15.17:0/1846879917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc419f8f0 con 0x7fbcc4102120 2024-03-24T03:00:20.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbcbb7fe640 1 -- 172.21.15.17:0/1846879917 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0018990 con 0x7fbcc4102120 2024-03-24T03:00:20.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.260+0000 7fbccc67e640 1 -- 172.21.15.17:0/1846879917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc419fe40 con 0x7fbcc4102120 2024-03-24T03:00:20.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.261+0000 7fbccc67e640 1 -- 172.21.15.17:0/1846879917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc40fd620 con 0x7fbcc4102120 2024-03-24T03:00:20.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.262+0000 7fbcbb7fe640 1 -- 172.21.15.17:0/1846879917 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbcc0010ac0 con 0x7fbcc4102120 2024-03-24T03:00:20.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.263+0000 7fbcbb7fe640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbcb00a2800 0x7fbcb00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:20.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.263+0000 7fbcbb7fe640 1 -- 172.21.15.17:0/1846879917 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fbcc00cf0e0 con 0x7fbcc4102120 2024-03-24T03:00:20.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.263+0000 7fbcc9c19640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbcb00a2800 0x7fbcb00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:20.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.263+0000 7fbcc9c19640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbcb00a2800 0x7fbcb00a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4198ef0 tx=0x7fbcb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:20.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.267+0000 7fbcbb7fe640 1 -- 172.21.15.17:0/1846879917 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbcc009b800 con 0x7fbcc4102120 2024-03-24T03:00:20.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.450+0000 7fbccc67e640 1 -- 172.21.15.17:0/1846879917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbcc40622d0 con 0x7fbcc4102120 2024-03-24T03:00:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.450+0000 7fbcbb7fe640 1 -- 172.21.15.17:0/1846879917 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbcc009f8a0 con 0x7fbcc4102120 2024-03-24T03:00:20.450 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T03:00:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.452+0000 7fbccc67e640 1 -- 172.21.15.17:0/1846879917 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbcb00a2800 msgr2=0x7fbcb00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.452+0000 7fbccc67e640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbcb00a2800 0x7fbcb00a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4198ef0 tx=0x7fbcb4005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.452+0000 7fbccc67e640 1 -- 172.21.15.17:0/1846879917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcc4102120 msgr2=0x7fbcc41979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.452+0000 7fbccc67e640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcc4102120 0x7fbcc41979e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000e9f0 tx=0x7fbcc000eeb0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.452+0000 7fbccc67e640 1 -- 172.21.15.17:0/1846879917 shutdown_connections 2024-03-24T03:00:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.452+0000 7fbccc67e640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbcb00a2800 0x7fbcb00a4cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.452+0000 7fbccc67e640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc4107380 0x7fbcc419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.452+0000 7fbccc67e640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc4104a50 0x7fbcc4197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.452+0000 7fbccc67e640 1 --2- 172.21.15.17:0/1846879917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcc4102120 0x7fbcc41979e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.452+0000 7fbccc67e640 1 -- 172.21.15.17:0/1846879917 >> 172.21.15.17:0/1846879917 conn(0x7fbcc40fbb30 msgr2=0x7fbcc41029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.452+0000 7fbccc67e640 1 -- 172.21.15.17:0/1846879917 shutdown_connections 2024-03-24T03:00:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.452+0000 7fbccc67e640 1 -- 172.21.15.17:0/1846879917 wait complete. 2024-03-24T03:00:20.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T03:00:20.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.567+0000 7fb654e68640 1 -- 172.21.15.17:0/294946176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb650100850 msgr2=0x7fb650100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.567+0000 7fb654e68640 1 --2- 172.21.15.17:0/294946176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb650100850 0x7fb650100cd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb640009900 tx=0x7fb64002bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:20.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.567+0000 7fb654e68640 1 -- 172.21.15.17:0/294946176 shutdown_connections 2024-03-24T03:00:20.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.567+0000 7fb654e68640 1 --2- 172.21.15.17:0/294946176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb650101210 0x7fb650108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.567+0000 7fb654e68640 1 --2- 172.21.15.17:0/294946176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb650100850 0x7fb650100cd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.567+0000 7fb654e68640 1 --2- 172.21.15.17:0/294946176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb650101c00 0x7fb650102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.567+0000 7fb654e68640 1 -- 172.21.15.17:0/294946176 >> 172.21.15.17:0/294946176 conn(0x7fb6500fb2e0 msgr2=0x7fb6500fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.567+0000 7fb654e68640 1 -- 172.21.15.17:0/294946176 shutdown_connections 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.567+0000 7fb654e68640 1 -- 172.21.15.17:0/294946176 wait complete. 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.567+0000 7fb654e68640 1 Processor -- start 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb654e68640 1 -- start start 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb654e68640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb650100850 0x7fb650199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb654e68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb650101210 0x7fb65019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb654e68640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb650101c00 0x7fb6501a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb654e68640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65010c290 con 0x7fb650100850 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb654e68640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65010c110 con 0x7fb650101210 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb654e68640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65010c410 con 0x7fb650101c00 2024-03-24T03:00:20.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb64e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb650100850 0x7fb650199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:20.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb64e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb650100850 0x7fb650199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53184/0 (socket says 172.21.15.17:53184) 2024-03-24T03:00:20.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb64e59c640 1 -- 172.21.15.17:0/1124860083 learned_addr learned my addr 172.21.15.17:0/1124860083 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:20.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb64dd9b640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb650101210 0x7fb65019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:20.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb64ed9d640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb650101c00 0x7fb6501a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:20.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb64e59c640 1 -- 172.21.15.17:0/1124860083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb650101c00 msgr2=0x7fb6501a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb64e59c640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb650101c00 0x7fb6501a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb64e59c640 1 -- 172.21.15.17:0/1124860083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb650101210 msgr2=0x7fb65019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb64e59c640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb650101210 0x7fb65019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb64e59c640 1 -- 172.21.15.17:0/1124860083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6501a18d0 con 0x7fb650100850 2024-03-24T03:00:20.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb64e59c640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb650100850 0x7fb650199c90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e9c0 tx=0x7fb63800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:20.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.568+0000 7fb6477fe640 1 -- 172.21.15.17:0/1124860083 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63800ccc0 con 0x7fb650100850 2024-03-24T03:00:20.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.569+0000 7fb6477fe640 1 -- 172.21.15.17:0/1124860083 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6380035f0 con 0x7fb650100850 2024-03-24T03:00:20.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.569+0000 7fb654e68640 1 -- 172.21.15.17:0/1124860083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6501a1bc0 con 0x7fb650100850 2024-03-24T03:00:20.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.569+0000 7fb6477fe640 1 -- 172.21.15.17:0/1124860083 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63800ce60 con 0x7fb650100850 2024-03-24T03:00:20.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.569+0000 7fb654e68640 1 -- 172.21.15.17:0/1124860083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6501a2090 con 0x7fb650100850 2024-03-24T03:00:20.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.569+0000 7fb654e68640 1 -- 172.21.15.17:0/1124860083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6501a2260 con 0x7fb650100850 2024-03-24T03:00:20.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.571+0000 7fb6477fe640 1 -- 172.21.15.17:0/1124860083 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb6380037a0 con 0x7fb650100850 2024-03-24T03:00:20.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.571+0000 7fb6477fe640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb63c0a2800 0x7fb63c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:20.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.571+0000 7fb64dd9b640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb63c0a2800 0x7fb63c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:20.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.571+0000 7fb6477fe640 1 -- 172.21.15.17:0/1124860083 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fb6380cf130 con 0x7fb650100850 2024-03-24T03:00:20.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.572+0000 7fb64dd9b640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb63c0a2800 0x7fb63c0a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb64002efd0 tx=0x7fb640000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:20.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.576+0000 7fb6477fe640 1 -- 172.21.15.17:0/1124860083 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb63809be40 con 0x7fb650100850 2024-03-24T03:00:20.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.797+0000 7fb654e68640 1 -- 172.21.15.17:0/1124860083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6501a2430 con 0x7fb650100850 2024-03-24T03:00:20.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.798+0000 7fb6477fe640 1 -- 172.21.15.17:0/1124860083 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11452 (secure 0 0 0) 0x7fb63809b740 con 0x7fb650100850 2024-03-24T03:00:20.797 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:20.797 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T02:58:36.476051+0000","modified":"2024-03-24T02:58:59.074424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24529,"mds_1":24521,"mds_2":24511,"mds_3":24493,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6841/1592961709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":1592961709},{"type":"v1","addr":"172.21.15.17:6841","nonce":1592961709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6839/2102503456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2102503456},{"type":"v1","addr":"172.21.15.19:6839","nonce":2102503456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24511,24529,24485,24493,24521]},"id":1}]} 2024-03-24T03:00:20.798 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 28 2024-03-24T03:00:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.800+0000 7fb654e68640 1 -- 172.21.15.17:0/1124860083 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb63c0a2800 msgr2=0x7fb63c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.800+0000 7fb654e68640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb63c0a2800 0x7fb63c0a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb64002efd0 tx=0x7fb640000bc0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.800+0000 7fb654e68640 1 -- 172.21.15.17:0/1124860083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb650100850 msgr2=0x7fb650199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.800+0000 7fb654e68640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb650100850 0x7fb650199c90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e9c0 tx=0x7fb63800edc0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.800+0000 7fb654e68640 1 -- 172.21.15.17:0/1124860083 shutdown_connections 2024-03-24T03:00:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.800+0000 7fb654e68640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb63c0a2800 0x7fb63c0a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.800+0000 7fb654e68640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb650101c00 0x7fb6501a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.800+0000 7fb654e68640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb650101210 0x7fb65019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.800+0000 7fb654e68640 1 --2- 172.21.15.17:0/1124860083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb650100850 0x7fb650199c90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.800+0000 7fb654e68640 1 -- 172.21.15.17:0/1124860083 >> 172.21.15.17:0/1124860083 conn(0x7fb6500fb2e0 msgr2=0x7fb6500ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.800+0000 7fb654e68640 1 -- 172.21.15.17:0/1124860083 shutdown_connections 2024-03-24T03:00:20.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.800+0000 7fb654e68640 1 -- 172.21.15.17:0/1124860083 wait complete. 2024-03-24T03:00:20.811 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T03:00:20.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.914+0000 7f96aa285640 1 -- 172.21.15.17:0/3437462428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a41017d0 msgr2=0x7f96a4101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.914+0000 7f96aa285640 1 --2- 172.21.15.17:0/3437462428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a41017d0 0x7f96a4101c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9694009860 tx=0x7f969402bb70 comp rx=0 tx=0).stop 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.914+0000 7f96aa285640 1 -- 172.21.15.17:0/3437462428 shutdown_connections 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.914+0000 7f96aa285640 1 --2- 172.21.15.17:0/3437462428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a4102190 0x7f96a4108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.914+0000 7f96aa285640 1 --2- 172.21.15.17:0/3437462428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a41017d0 0x7f96a4101c50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.914+0000 7f96aa285640 1 --2- 172.21.15.17:0/3437462428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a41005e0 0x7f96a4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.914+0000 7f96aa285640 1 -- 172.21.15.17:0/3437462428 >> 172.21.15.17:0/3437462428 conn(0x7f96a40fbb50 msgr2=0x7f96a40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.914+0000 7f96aa285640 1 -- 172.21.15.17:0/3437462428 shutdown_connections 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.914+0000 7f96aa285640 1 -- 172.21.15.17:0/3437462428 wait complete. 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96aa285640 1 Processor -- start 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96aa285640 1 -- start start 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96aa285640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a41005e0 0x7f96a406a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96aa285640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a41017d0 0x7f96a419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:20.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96aa285640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a4102190 0x7f96a41a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96aa285640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a410c260 con 0x7f96a41017d0 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96aa285640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a410c0e0 con 0x7f96a4102190 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96aa285640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a410c3e0 con 0x7f96a41005e0 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a41017d0 0x7f96a419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a41017d0 0x7f96a419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35374/0 (socket says 172.21.15.17:35374) 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96a8822640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a4102190 0x7f96a41a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96a37fe640 1 -- 172.21.15.17:0/853450017 learned_addr learned my addr 172.21.15.17:0/853450017 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96a3fff640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a41005e0 0x7f96a406a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.915+0000 7f96a37fe640 1 -- 172.21.15.17:0/853450017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a41005e0 msgr2=0x7f96a406a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.916+0000 7f96a37fe640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a41005e0 0x7f96a406a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.916+0000 7f96a37fe640 1 -- 172.21.15.17:0/853450017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a4102190 msgr2=0x7f96a41a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.916+0000 7f96a37fe640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a4102190 0x7f96a41a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.916+0000 7f96a37fe640 1 -- 172.21.15.17:0/853450017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a41a1880 con 0x7f96a41017d0 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.916+0000 7f96a37fe640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a41017d0 0x7f96a419eb60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f96940349e0 tx=0x7f9694032450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:20.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.916+0000 7f96a17fa640 1 -- 172.21.15.17:0/853450017 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694041030 con 0x7f96a41017d0 2024-03-24T03:00:20.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.916+0000 7f96aa285640 1 -- 172.21.15.17:0/853450017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a41a1b10 con 0x7f96a41017d0 2024-03-24T03:00:20.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.916+0000 7f96a17fa640 1 -- 172.21.15.17:0/853450017 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f969402c510 con 0x7f96a41017d0 2024-03-24T03:00:20.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.916+0000 7f96a17fa640 1 -- 172.21.15.17:0/853450017 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694039040 con 0x7f96a41017d0 2024-03-24T03:00:20.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.916+0000 7f96aa285640 1 -- 172.21.15.17:0/853450017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a41a2050 con 0x7f96a41017d0 2024-03-24T03:00:20.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.917+0000 7f96aa285640 1 -- 172.21.15.17:0/853450017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a4107950 con 0x7f96a41017d0 2024-03-24T03:00:20.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.918+0000 7f96a17fa640 1 -- 172.21.15.17:0/853450017 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9694044070 con 0x7f96a41017d0 2024-03-24T03:00:20.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.919+0000 7f96a17fa640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96900a2800 0x7f96900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:20.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.919+0000 7f96a3fff640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96900a2800 0x7f96900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:20.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.919+0000 7f96a17fa640 1 -- 172.21.15.17:0/853450017 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f96940f0a80 con 0x7f96a41017d0 2024-03-24T03:00:20.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.919+0000 7f96a3fff640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96900a2800 0x7f96900a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96a4101630 tx=0x7f968c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:20.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:20.922+0000 7f96a17fa640 1 -- 172.21.15.17:0/853450017 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f96940bd790 con 0x7f96a41017d0 2024-03-24T03:00:21.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:20 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3603240631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:00:21.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:20 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3842415167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:00:21.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:20 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1846879917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:00:21.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.110+0000 7f96aa285640 1 -- 172.21.15.17:0/853450017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96a402d070 con 0x7f96a41017d0 2024-03-24T03:00:21.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.112+0000 7f96a17fa640 1 -- 172.21.15.17:0/853450017 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24514 (secure 0 0 0) 0x7f96940bd090 con 0x7f96a41017d0 2024-03-24T03:00:21.111 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:21.112 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":91,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:59:30.964668+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T03:00:21.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.114+0000 7f96aa285640 1 -- 172.21.15.17:0/853450017 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96900a2800 msgr2=0x7f96900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.114+0000 7f96aa285640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96900a2800 0x7f96900a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96a4101630 tx=0x7f968c005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:21.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.114+0000 7f96aa285640 1 -- 172.21.15.17:0/853450017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a41017d0 msgr2=0x7f96a419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.114+0000 7f96aa285640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a41017d0 0x7f96a419eb60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f96940349e0 tx=0x7f9694032450 comp rx=0 tx=0).stop 2024-03-24T03:00:21.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.114+0000 7f96aa285640 1 -- 172.21.15.17:0/853450017 shutdown_connections 2024-03-24T03:00:21.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.114+0000 7f96aa285640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f96900a2800 0x7f96900a4cc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.114+0000 7f96aa285640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a4102190 0x7f96a41a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.114+0000 7f96aa285640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a41017d0 0x7f96a419eb60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.114+0000 7f96aa285640 1 --2- 172.21.15.17:0/853450017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a41005e0 0x7f96a406a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.114+0000 7f96aa285640 1 -- 172.21.15.17:0/853450017 >> 172.21.15.17:0/853450017 conn(0x7f96a40fbb50 msgr2=0x7f96a4104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:21.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.114+0000 7f96aa285640 1 -- 172.21.15.17:0/853450017 shutdown_connections 2024-03-24T03:00:21.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.114+0000 7f96aa285640 1 -- 172.21.15.17:0/853450017 wait complete. 2024-03-24T03:00:21.124 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:20 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3603240631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:00:21.124 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:20 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3842415167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:00:21.124 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:20 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1846879917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:00:21.125 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T02:53:54.207139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-24T02:58:34.785192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-24T02:58:35.679772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-24T02:58:39.221772+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-24T03:00:21.126 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T03:00:21.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.232+0000 7f1e0f376640 1 -- 172.21.15.17:0/721322246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e08107380 msgr2=0x7f1e081097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.232+0000 7f1e0f376640 1 --2- 172.21.15.17:0/721322246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e08107380 0x7f1e081097d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009ca0 tx=0x7f1e0002c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.232+0000 7f1e0f376640 1 -- 172.21.15.17:0/721322246 shutdown_connections 2024-03-24T03:00:21.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.232+0000 7f1e0f376640 1 --2- 172.21.15.17:0/721322246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e08107380 0x7f1e081097d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.232+0000 7f1e0f376640 1 --2- 172.21.15.17:0/721322246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e08104a50 0x7f1e08106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.232+0000 7f1e0f376640 1 --2- 172.21.15.17:0/721322246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e08102120 0x7f1e08104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.232+0000 7f1e0f376640 1 -- 172.21.15.17:0/721322246 >> 172.21.15.17:0/721322246 conn(0x7f1e080fbb30 msgr2=0x7f1e080fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:21.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.232+0000 7f1e0f376640 1 -- 172.21.15.17:0/721322246 shutdown_connections 2024-03-24T03:00:21.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.232+0000 7f1e0f376640 1 -- 172.21.15.17:0/721322246 wait complete. 2024-03-24T03:00:21.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0f376640 1 Processor -- start 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0f376640 1 -- start start 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0f376640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e08102120 0x7f1e081979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0f376640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e08104a50 0x7f1e08197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0f376640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e08107380 0x7f1e0819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0f376640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0810d320 con 0x7f1e08104a50 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0f376640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0810d1a0 con 0x7f1e08107380 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0f376640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0810d4a0 con 0x7f1e08102120 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0c911640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e08104a50 0x7f1e08197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0c911640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e08104a50 0x7f1e08197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35402/0 (socket says 172.21.15.17:35402) 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0c911640 1 -- 172.21.15.17:0/1779102274 learned_addr learned my addr 172.21.15.17:0/1779102274 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0d913640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e08107380 0x7f1e0819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.233+0000 7f1e0d112640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e08102120 0x7f1e081979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.234+0000 7f1e0c911640 1 -- 172.21.15.17:0/1779102274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e08102120 msgr2=0x7f1e081979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.234+0000 7f1e0c911640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e08102120 0x7f1e081979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.234+0000 7f1e0c911640 1 -- 172.21.15.17:0/1779102274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e08107380 msgr2=0x7f1e0819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.234+0000 7f1e0c911640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e08107380 0x7f1e0819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.234+0000 7f1e0c911640 1 -- 172.21.15.17:0/1779102274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0819f5e0 con 0x7f1e08104a50 2024-03-24T03:00:21.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.234+0000 7f1e0d913640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e08107380 0x7f1e0819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:21.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.234+0000 7f1e0c911640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e08104a50 0x7f1e08197f00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9c0 tx=0x7f1df800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:21.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.234+0000 7f1dfe7fc640 1 -- 172.21.15.17:0/1779102274 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df800cd80 con 0x7f1e08104a50 2024-03-24T03:00:21.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.234+0000 7f1dfe7fc640 1 -- 172.21.15.17:0/1779102274 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df8005500 con 0x7f1e08104a50 2024-03-24T03:00:21.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.234+0000 7f1e0f376640 1 -- 172.21.15.17:0/1779102274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0819f8d0 con 0x7f1e08104a50 2024-03-24T03:00:21.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.234+0000 7f1dfe7fc640 1 -- 172.21.15.17:0/1779102274 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8010690 con 0x7f1e08104a50 2024-03-24T03:00:21.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.234+0000 7f1e0f376640 1 -- 172.21.15.17:0/1779102274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0819fe20 con 0x7f1e08104a50 2024-03-24T03:00:21.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.235+0000 7f1e0f376640 1 -- 172.21.15.17:0/1779102274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e080fd620 con 0x7f1e08104a50 2024-03-24T03:00:21.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.236+0000 7f1dfe7fc640 1 -- 172.21.15.17:0/1779102274 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1df80046c0 con 0x7f1e08104a50 2024-03-24T03:00:21.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.237+0000 7f1dfe7fc640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1df40a2800 0x7f1df40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:21.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.237+0000 7f1e0d112640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1df40a2800 0x7f1df40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:21.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.237+0000 7f1dfe7fc640 1 -- 172.21.15.17:0/1779102274 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f1df80cf170 con 0x7f1e08104a50 2024-03-24T03:00:21.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.237+0000 7f1e0d112640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1df40a2800 0x7f1df40a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009bd0 tx=0x7f1e04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:21.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.241+0000 7f1dfe7fc640 1 -- 172.21.15.17:0/1779102274 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1df809be80 con 0x7f1e08104a50 2024-03-24T03:00:21.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:20 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3603240631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:00:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:20 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3842415167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:00:21.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:20 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1846879917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:00:21.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.423+0000 7f1e0f376640 1 -- 172.21.15.17:0/1779102274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e080622a0 con 0x7f1e08104a50 2024-03-24T03:00:21.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.424+0000 7f1dfe7fc640 1 -- 172.21.15.17:0/1779102274 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1df809b780 con 0x7f1e08104a50 2024-03-24T03:00:21.423 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-24T03:00:21.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.426+0000 7f1e0f376640 1 -- 172.21.15.17:0/1779102274 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1df40a2800 msgr2=0x7f1df40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.426+0000 7f1e0f376640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1df40a2800 0x7f1df40a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009bd0 tx=0x7f1e04005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.426+0000 7f1e0f376640 1 -- 172.21.15.17:0/1779102274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e08104a50 msgr2=0x7f1e08197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.426+0000 7f1e0f376640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e08104a50 0x7f1e08197f00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9c0 tx=0x7f1df800ee80 comp rx=0 tx=0).stop 2024-03-24T03:00:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.426+0000 7f1e0f376640 1 -- 172.21.15.17:0/1779102274 shutdown_connections 2024-03-24T03:00:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.426+0000 7f1e0f376640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1df40a2800 0x7f1df40a4cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.426+0000 7f1e0f376640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e08107380 0x7f1e0819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.426+0000 7f1e0f376640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e08104a50 0x7f1e08197f00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.426+0000 7f1e0f376640 1 --2- 172.21.15.17:0/1779102274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e08102120 0x7f1e081979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.426+0000 7f1e0f376640 1 -- 172.21.15.17:0/1779102274 >> 172.21.15.17:0/1779102274 conn(0x7f1e080fbb30 msgr2=0x7f1e08102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.426+0000 7f1e0f376640 1 -- 172.21.15.17:0/1779102274 shutdown_connections 2024-03-24T03:00:21.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.426+0000 7f1e0f376640 1 -- 172.21.15.17:0/1779102274 wait complete. 2024-03-24T03:00:21.436 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-24T03:00:21.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.543+0000 7f028249a640 1 -- 172.21.15.17:0/447195052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f027c102130 msgr2=0x7f027c104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.543+0000 7f028249a640 1 --2- 172.21.15.17:0/447195052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f027c102130 0x7f027c104520 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0264009900 tx=0x7f026402bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:21.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.543+0000 7f028249a640 1 -- 172.21.15.17:0/447195052 shutdown_connections 2024-03-24T03:00:21.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.543+0000 7f028249a640 1 --2- 172.21.15.17:0/447195052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027c107390 0x7f027c1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.543+0000 7f028249a640 1 --2- 172.21.15.17:0/447195052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c104a60 0x7f027c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.543+0000 7f028249a640 1 --2- 172.21.15.17:0/447195052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f027c102130 0x7f027c104520 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.543+0000 7f028249a640 1 -- 172.21.15.17:0/447195052 >> 172.21.15.17:0/447195052 conn(0x7f027c0fbb40 msgr2=0x7f027c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:21.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.543+0000 7f028249a640 1 -- 172.21.15.17:0/447195052 shutdown_connections 2024-03-24T03:00:21.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.543+0000 7f028249a640 1 -- 172.21.15.17:0/447195052 wait complete. 2024-03-24T03:00:21.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.543+0000 7f028249a640 1 Processor -- start 2024-03-24T03:00:21.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.543+0000 7f028249a640 1 -- start start 2024-03-24T03:00:21.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f028249a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c102130 0x7f027c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:21.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f028249a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027c104a60 0x7f027c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:21.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f028249a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f027c107390 0x7f027c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:21.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f028249a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c10d340 con 0x7f027c107390 2024-03-24T03:00:21.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f028249a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c10d1c0 con 0x7f027c104a60 2024-03-24T03:00:21.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f028249a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c10d4c0 con 0x7f027c102130 2024-03-24T03:00:21.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f0280a37640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f027c107390 0x7f027c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:21.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f0280a37640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f027c107390 0x7f027c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35430/0 (socket says 172.21.15.17:35430) 2024-03-24T03:00:21.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f0280a37640 1 -- 172.21.15.17:0/1133045564 learned_addr learned my addr 172.21.15.17:0/1133045564 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:21.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f027bfff640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c102130 0x7f027c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:21.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f027b7fe640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027c104a60 0x7f027c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:21.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f0280a37640 1 -- 172.21.15.17:0/1133045564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c102130 msgr2=0x7f027c197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f0280a37640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c102130 0x7f027c197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f0280a37640 1 -- 172.21.15.17:0/1133045564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027c104a60 msgr2=0x7f027c197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f0280a37640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027c104a60 0x7f027c197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f0280a37640 1 -- 172.21.15.17:0/1133045564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c19f670 con 0x7f027c107390 2024-03-24T03:00:21.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f0280a37640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f027c107390 0x7f027c19f000 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f027400efd0 tx=0x7f027400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:21.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.544+0000 7f02797fa640 1 -- 172.21.15.17:0/1133045564 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0274003b70 con 0x7f027c107390 2024-03-24T03:00:21.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.545+0000 7f02797fa640 1 -- 172.21.15.17:0/1133045564 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0274003d10 con 0x7f027c107390 2024-03-24T03:00:21.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.545+0000 7f028249a640 1 -- 172.21.15.17:0/1133045564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027c19f960 con 0x7f027c107390 2024-03-24T03:00:21.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.545+0000 7f02797fa640 1 -- 172.21.15.17:0/1133045564 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02740109f0 con 0x7f027c107390 2024-03-24T03:00:21.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.545+0000 7f028249a640 1 -- 172.21.15.17:0/1133045564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027c19fe60 con 0x7f027c107390 2024-03-24T03:00:21.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.546+0000 7f028249a640 1 -- 172.21.15.17:0/1133045564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f027c0fd630 con 0x7f027c107390 2024-03-24T03:00:21.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.547+0000 7f02797fa640 1 -- 172.21.15.17:0/1133045564 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0274004020 con 0x7f027c107390 2024-03-24T03:00:21.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.547+0000 7f02797fa640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f02680a2800 0x7f02680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:21.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.547+0000 7f027bfff640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f02680a2800 0x7f02680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:21.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.547+0000 7f02797fa640 1 -- 172.21.15.17:0/1133045564 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f02740cf200 con 0x7f027c107390 2024-03-24T03:00:21.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.548+0000 7f027bfff640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f02680a2800 0x7f02680a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f026402c7b0 tx=0x7f0264005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:21.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.551+0000 7f02797fa640 1 -- 172.21.15.17:0/1133045564 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f027409bf70 con 0x7f027c107390 2024-03-24T03:00:21.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.735+0000 7f028249a640 1 -- 172.21.15.17:0/1133045564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f027c0622a0 con 0x7f027c107390 2024-03-24T03:00:21.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.736+0000 7f02797fa640 1 -- 172.21.15.17:0/1133045564 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f027409b870 con 0x7f027c107390 2024-03-24T03:00:21.735 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T03:00:21.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.737+0000 7f028249a640 1 -- 172.21.15.17:0/1133045564 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f02680a2800 msgr2=0x7f02680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.737+0000 7f028249a640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f02680a2800 0x7f02680a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f026402c7b0 tx=0x7f0264005fd0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.737+0000 7f028249a640 1 -- 172.21.15.17:0/1133045564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f027c107390 msgr2=0x7f027c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.737+0000 7f028249a640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f027c107390 0x7f027c19f000 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f027400efd0 tx=0x7f027400c6a0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.738+0000 7f028249a640 1 -- 172.21.15.17:0/1133045564 shutdown_connections 2024-03-24T03:00:21.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.738+0000 7f028249a640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f02680a2800 0x7f02680a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.738+0000 7f028249a640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f027c107390 0x7f027c19f000 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.738+0000 7f028249a640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027c104a60 0x7f027c197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.738+0000 7f028249a640 1 --2- 172.21.15.17:0/1133045564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027c102130 0x7f027c197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.738+0000 7f028249a640 1 -- 172.21.15.17:0/1133045564 >> 172.21.15.17:0/1133045564 conn(0x7f027c0fbb40 msgr2=0x7f027c102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:21.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.738+0000 7f028249a640 1 -- 172.21.15.17:0/1133045564 shutdown_connections 2024-03-24T03:00:21.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.738+0000 7f028249a640 1 -- 172.21.15.17:0/1133045564 wait complete. 2024-03-24T03:00:21.748 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-24T03:00:21.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 -- 172.21.15.17:0/968203569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda88101790 msgr2=0x7fda88101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 --2- 172.21.15.17:0/968203569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda88101790 0x7fda88101c10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fda78009ca0 tx=0x7fda7802c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 -- 172.21.15.17:0/968203569 shutdown_connections 2024-03-24T03:00:21.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 --2- 172.21.15.17:0/968203569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88102150 0x7fda881089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 --2- 172.21.15.17:0/968203569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda88101790 0x7fda88101c10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 --2- 172.21.15.17:0/968203569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda881005a0 0x7fda881009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 -- 172.21.15.17:0/968203569 >> 172.21.15.17:0/968203569 conn(0x7fda880fbb30 msgr2=0x7fda880fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:21.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 -- 172.21.15.17:0/968203569 shutdown_connections 2024-03-24T03:00:21.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 -- 172.21.15.17:0/968203569 wait complete. 2024-03-24T03:00:21.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 Processor -- start 2024-03-24T03:00:21.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 -- start start 2024-03-24T03:00:21.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda881005a0 0x7fda881979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:21.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88101790 0x7fda88197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:21.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda88102150 0x7fda8819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:21.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8810c230 con 0x7fda881005a0 2024-03-24T03:00:21.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8810c0b0 con 0x7fda88101790 2024-03-24T03:00:21.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8ea75640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8810c3b0 con 0x7fda88102150 2024-03-24T03:00:21.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8c811640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda881005a0 0x7fda881979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:21.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8c811640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda881005a0 0x7fda881979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35444/0 (socket says 172.21.15.17:35444) 2024-03-24T03:00:21.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.856+0000 7fda8c811640 1 -- 172.21.15.17:0/3039524379 learned_addr learned my addr 172.21.15.17:0/3039524379 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:21.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda87fff640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88101790 0x7fda88197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:21.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda8d012640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda88102150 0x7fda8819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:21.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda8c811640 1 -- 172.21.15.17:0/3039524379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda88102150 msgr2=0x7fda8819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda8c811640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda88102150 0x7fda8819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda8c811640 1 -- 172.21.15.17:0/3039524379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88101790 msgr2=0x7fda88197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda8c811640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88101790 0x7fda88197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda8c811640 1 -- 172.21.15.17:0/3039524379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8819f5e0 con 0x7fda881005a0 2024-03-24T03:00:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda8c811640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda881005a0 0x7fda881979c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fda8000e9c0 tx=0x7fda8000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda85ffb640 1 -- 172.21.15.17:0/3039524379 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8000ccc0 con 0x7fda881005a0 2024-03-24T03:00:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda85ffb640 1 -- 172.21.15.17:0/3039524379 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda800035f0 con 0x7fda881005a0 2024-03-24T03:00:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda8ea75640 1 -- 172.21.15.17:0/3039524379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8819f8d0 con 0x7fda881005a0 2024-03-24T03:00:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda85ffb640 1 -- 172.21.15.17:0/3039524379 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8000ce60 con 0x7fda881005a0 2024-03-24T03:00:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.857+0000 7fda8ea75640 1 -- 172.21.15.17:0/3039524379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda8819fe40 con 0x7fda881005a0 2024-03-24T03:00:21.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.858+0000 7fda8ea75640 1 -- 172.21.15.17:0/3039524379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda88068490 con 0x7fda881005a0 2024-03-24T03:00:21.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.859+0000 7fda85ffb640 1 -- 172.21.15.17:0/3039524379 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fda800037a0 con 0x7fda881005a0 2024-03-24T03:00:21.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.860+0000 7fda85ffb640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda740a2880 0x7fda740a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:21.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.860+0000 7fda87fff640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda740a2880 0x7fda740a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:21.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.860+0000 7fda85ffb640 1 -- 172.21.15.17:0/3039524379 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fda800cf070 con 0x7fda881005a0 2024-03-24T03:00:21.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.860+0000 7fda87fff640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda740a2880 0x7fda740a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda78006010 tx=0x7fda78032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:21.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:21.864+0000 7fda85ffb640 1 -- 172.21.15.17:0/3039524379 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fda8009bde0 con 0x7fda881005a0 2024-03-24T03:00:22.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.049+0000 7fda8ea75640 1 -- 172.21.15.17:0/3039524379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda881a00d0 con 0x7fda881005a0 2024-03-24T03:00:22.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.050+0000 7fda85ffb640 1 -- 172.21.15.17:0/3039524379 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda800860f0 con 0x7fda881005a0 2024-03-24T03:00:22.049 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-24T03:00:22.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.052+0000 7fda8ea75640 1 -- 172.21.15.17:0/3039524379 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda740a2880 msgr2=0x7fda740a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.052+0000 7fda8ea75640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda740a2880 0x7fda740a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda78006010 tx=0x7fda78032590 comp rx=0 tx=0).stop 2024-03-24T03:00:22.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.052+0000 7fda8ea75640 1 -- 172.21.15.17:0/3039524379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda881005a0 msgr2=0x7fda881979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.052+0000 7fda8ea75640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda881005a0 0x7fda881979c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fda8000e9c0 tx=0x7fda8000edc0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.052+0000 7fda8ea75640 1 -- 172.21.15.17:0/3039524379 shutdown_connections 2024-03-24T03:00:22.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.052+0000 7fda8ea75640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda740a2880 0x7fda740a4d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.052+0000 7fda8ea75640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda88102150 0x7fda8819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.052+0000 7fda8ea75640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88101790 0x7fda88197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.052+0000 7fda8ea75640 1 --2- 172.21.15.17:0/3039524379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda881005a0 0x7fda881979c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.052+0000 7fda8ea75640 1 -- 172.21.15.17:0/3039524379 >> 172.21.15.17:0/3039524379 conn(0x7fda880fbb30 msgr2=0x7fda880fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:22.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.052+0000 7fda8ea75640 1 -- 172.21.15.17:0/3039524379 shutdown_connections 2024-03-24T03:00:22.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.052+0000 7fda8ea75640 1 -- 172.21.15.17:0/3039524379 wait complete. 2024-03-24T03:00:22.061 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:21 smithi017 ceph-mon[31926]: pgmap v346: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-24T03:00:22.061 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:21 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1124860083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:22.062 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:21 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/853450017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:22.062 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:21 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1779102274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:00:22.062 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:21 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1133045564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:00:22.065 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-24T03:00:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:21 smithi019 ceph-mon[35944]: pgmap v346: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-24T03:00:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:21 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1124860083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:21 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/853450017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:21 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1779102274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:00:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:21 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1133045564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:00:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.171+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/989195669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a941017a0 msgr2=0x7f1a94101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.171+0000 7f1a9c0e8640 1 --2- 172.21.15.17:0/989195669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a941017a0 0x7f1a94101c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009860 tx=0x7f1a8402bb70 comp rx=0 tx=0).stop 2024-03-24T03:00:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.171+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/989195669 shutdown_connections 2024-03-24T03:00:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.171+0000 7f1a9c0e8640 1 --2- 172.21.15.17:0/989195669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a94102160 0x7f1a941089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.171+0000 7f1a9c0e8640 1 --2- 172.21.15.17:0/989195669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a941017a0 0x7f1a94101c20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.171+0000 7f1a9c0e8640 1 --2- 172.21.15.17:0/989195669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a941005b0 0x7f1a941009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.171+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/989195669 >> 172.21.15.17:0/989195669 conn(0x7f1a940fbb40 msgr2=0x7f1a940fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.171+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/989195669 shutdown_connections 2024-03-24T03:00:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.171+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/989195669 wait complete. 2024-03-24T03:00:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a9c0e8640 1 Processor -- start 2024-03-24T03:00:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a9c0e8640 1 -- start start 2024-03-24T03:00:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a9c0e8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a941005b0 0x7f1a94197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a9c0e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a941017a0 0x7f1a94197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a9c0e8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a94102160 0x7f1a9419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a9c0e8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9410c230 con 0x7f1a941005b0 2024-03-24T03:00:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a9c0e8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9410c0b0 con 0x7f1a941017a0 2024-03-24T03:00:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a9c0e8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9410c3b0 con 0x7f1a94102160 2024-03-24T03:00:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a99e84640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a941005b0 0x7f1a94197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a99e84640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a941005b0 0x7f1a94197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35464/0 (socket says 172.21.15.17:35464) 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a99683640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a941017a0 0x7f1a94197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a99e84640 1 -- 172.21.15.17:0/2229748220 learned_addr learned my addr 172.21.15.17:0/2229748220 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a9a685640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a94102160 0x7f1a9419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a99e84640 1 -- 172.21.15.17:0/2229748220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a94102160 msgr2=0x7f1a9419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a99e84640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a94102160 0x7f1a9419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a99e84640 1 -- 172.21.15.17:0/2229748220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a941017a0 msgr2=0x7f1a94197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a99e84640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a941017a0 0x7f1a94197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a99e84640 1 -- 172.21.15.17:0/2229748220 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9419f630 con 0x7f1a941005b0 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.172+0000 7f1a99e84640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a941005b0 0x7f1a94197a80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000e9a0 tx=0x7f1a9000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.173+0000 7f1a8affd640 1 -- 172.21.15.17:0/2229748220 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9000cd80 con 0x7f1a941005b0 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.173+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/2229748220 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9419f920 con 0x7f1a941005b0 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.173+0000 7f1a8affd640 1 -- 172.21.15.17:0/2229748220 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a90010910 con 0x7f1a941005b0 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.173+0000 7f1a8affd640 1 -- 172.21.15.17:0/2229748220 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90018a40 con 0x7f1a941005b0 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.173+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/2229748220 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9419fe90 con 0x7f1a941005b0 2024-03-24T03:00:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.174+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/2229748220 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a941a0120 con 0x7f1a941005b0 2024-03-24T03:00:22.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.175+0000 7f1a8affd640 1 -- 172.21.15.17:0/2229748220 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1a90010ac0 con 0x7f1a941005b0 2024-03-24T03:00:22.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.175+0000 7f1a8affd640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1a800a2800 0x7f1a800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:22.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.175+0000 7f1a8affd640 1 -- 172.21.15.17:0/2229748220 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f1a900cf0d0 con 0x7f1a941005b0 2024-03-24T03:00:22.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.175+0000 7f1a99683640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1a800a2800 0x7f1a800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:22.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.176+0000 7f1a99683640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1a800a2800 0x7f1a800a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1a8402c180 tx=0x7f1a84034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:22.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.180+0000 7f1a8affd640 1 -- 172.21.15.17:0/2229748220 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1a90053740 con 0x7f1a941005b0 2024-03-24T03:00:22.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:21 smithi188 ceph-mon[36206]: pgmap v346: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-24T03:00:22.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:21 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1124860083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:22.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:21 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/853450017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:22.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:21 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1779102274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:00:22.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:21 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1133045564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:00:22.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.364+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/2229748220 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1a941a02f0 con 0x7f1a941005b0 2024-03-24T03:00:22.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.364+0000 7f1a8affd640 1 -- 172.21.15.17:0/2229748220 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1a9009be40 con 0x7f1a941005b0 2024-03-24T03:00:22.363 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T03:00:22.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.366+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/2229748220 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1a800a2800 msgr2=0x7f1a800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.366+0000 7f1a9c0e8640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1a800a2800 0x7f1a800a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1a8402c180 tx=0x7f1a84034ad0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.366+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/2229748220 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a941005b0 msgr2=0x7f1a94197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.366+0000 7f1a9c0e8640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a941005b0 0x7f1a94197a80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000e9a0 tx=0x7f1a9000ee60 comp rx=0 tx=0).stop 2024-03-24T03:00:22.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.366+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/2229748220 shutdown_connections 2024-03-24T03:00:22.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.366+0000 7f1a9c0e8640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f1a800a2800 0x7f1a800a4cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.366+0000 7f1a9c0e8640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a94102160 0x7f1a9419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.366+0000 7f1a9c0e8640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a941017a0 0x7f1a94197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.366+0000 7f1a9c0e8640 1 --2- 172.21.15.17:0/2229748220 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a941005b0 0x7f1a94197a80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.366+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/2229748220 >> 172.21.15.17:0/2229748220 conn(0x7f1a940fbb40 msgr2=0x7f1a940fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:22.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.366+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/2229748220 shutdown_connections 2024-03-24T03:00:22.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.366+0000 7f1a9c0e8640 1 -- 172.21.15.17:0/2229748220 wait complete. 2024-03-24T03:00:22.377 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T03:00:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.480+0000 7efff3d40640 1 -- 172.21.15.17:0/3036603060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effec1021c0 msgr2=0x7effec1045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.480+0000 7efff3d40640 1 --2- 172.21.15.17:0/3036603060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effec1021c0 0x7effec1045b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7effe8009860 tx=0x7effe802bb70 comp rx=0 tx=0).stop 2024-03-24T03:00:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.480+0000 7efff3d40640 1 -- 172.21.15.17:0/3036603060 shutdown_connections 2024-03-24T03:00:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.480+0000 7efff3d40640 1 --2- 172.21.15.17:0/3036603060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effec107420 0x7effec109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.480+0000 7efff3d40640 1 --2- 172.21.15.17:0/3036603060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effec104af0 0x7effec106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.480+0000 7efff3d40640 1 --2- 172.21.15.17:0/3036603060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effec1021c0 0x7effec1045b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.480+0000 7efff3d40640 1 -- 172.21.15.17:0/3036603060 >> 172.21.15.17:0/3036603060 conn(0x7effec0fbbd0 msgr2=0x7effec0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.480+0000 7efff3d40640 1 -- 172.21.15.17:0/3036603060 shutdown_connections 2024-03-24T03:00:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.480+0000 7efff3d40640 1 -- 172.21.15.17:0/3036603060 wait complete. 2024-03-24T03:00:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff3d40640 1 Processor -- start 2024-03-24T03:00:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff3d40640 1 -- start start 2024-03-24T03:00:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff3d40640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effec1021c0 0x7effec1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff3d40640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effec104af0 0x7effec197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff3d40640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effec107420 0x7effec19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff3d40640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec10d370 con 0x7effec104af0 2024-03-24T03:00:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff3d40640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec10d1f0 con 0x7effec1021c0 2024-03-24T03:00:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff3d40640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec10d4f0 con 0x7effec107420 2024-03-24T03:00:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff12db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effec104af0 0x7effec197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff1adc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effec1021c0 0x7effec1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff12db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effec104af0 0x7effec197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35494/0 (socket says 172.21.15.17:35494) 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff12db640 1 -- 172.21.15.17:0/1041274541 learned_addr learned my addr 172.21.15.17:0/1041274541 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff22dd640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effec107420 0x7effec19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff12db640 1 -- 172.21.15.17:0/1041274541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effec107420 msgr2=0x7effec19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff12db640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effec107420 0x7effec19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff12db640 1 -- 172.21.15.17:0/1041274541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effec1021c0 msgr2=0x7effec1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff12db640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effec1021c0 0x7effec1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff12db640 1 -- 172.21.15.17:0/1041274541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec19f600 con 0x7effec104af0 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff1adc640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effec1021c0 0x7effec1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.481+0000 7efff12db640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effec104af0 0x7effec197f20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7effdc00e9f0 tx=0x7effdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.482+0000 7effe2ffd640 1 -- 172.21.15.17:0/1041274541 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc00ccc0 con 0x7effec104af0 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.482+0000 7effe2ffd640 1 -- 172.21.15.17:0/1041274541 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effdc005500 con 0x7effec104af0 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.482+0000 7efff3d40640 1 -- 172.21.15.17:0/1041274541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effec19f8f0 con 0x7effec104af0 2024-03-24T03:00:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.482+0000 7effe2ffd640 1 -- 172.21.15.17:0/1041274541 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc010690 con 0x7effec104af0 2024-03-24T03:00:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.482+0000 7efff3d40640 1 -- 172.21.15.17:0/1041274541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec19fdf0 con 0x7effec104af0 2024-03-24T03:00:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.483+0000 7efff3d40640 1 -- 172.21.15.17:0/1041274541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effec0fd6c0 con 0x7effec104af0 2024-03-24T03:00:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.484+0000 7effe2ffd640 1 -- 172.21.15.17:0/1041274541 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7effdc0046c0 con 0x7effec104af0 2024-03-24T03:00:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.484+0000 7effe2ffd640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7effd80a2800 0x7effd80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.484+0000 7efff1adc640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7effd80a2800 0x7effd80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.484+0000 7effe2ffd640 1 -- 172.21.15.17:0/1041274541 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7effdc0cf2d0 con 0x7effec104af0 2024-03-24T03:00:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.485+0000 7efff1adc640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7effd80a2800 0x7effd80a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7effe802c180 tx=0x7effe802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.488+0000 7effe2ffd640 1 -- 172.21.15.17:0/1041274541 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7effdc053740 con 0x7effec104af0 2024-03-24T03:00:22.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.712+0000 7efff3d40640 1 -- 172.21.15.17:0/1041274541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effec02d2c0 con 0x7effec104af0 2024-03-24T03:00:22.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.712+0000 7effe2ffd640 1 -- 172.21.15.17:0/1041274541 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11452 (secure 0 0 0) 0x7effdc09c040 con 0x7effec104af0 2024-03-24T03:00:22.712 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:22.712 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T02:58:36.476051+0000","modified":"2024-03-24T02:58:59.074424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24529,"mds_1":24521,"mds_2":24511,"mds_3":24493,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6841/1592961709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":1592961709},{"type":"v1","addr":"172.21.15.17:6841","nonce":1592961709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6839/2102503456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2102503456},{"type":"v1","addr":"172.21.15.19:6839","nonce":2102503456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24511,24529,24485,24493,24521]},"id":1}]} 2024-03-24T03:00:22.712 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 28 2024-03-24T03:00:22.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.714+0000 7efff3d40640 1 -- 172.21.15.17:0/1041274541 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7effd80a2800 msgr2=0x7effd80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.714+0000 7efff3d40640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7effd80a2800 0x7effd80a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7effe802c180 tx=0x7effe802e040 comp rx=0 tx=0).stop 2024-03-24T03:00:22.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.714+0000 7efff3d40640 1 -- 172.21.15.17:0/1041274541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effec104af0 msgr2=0x7effec197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.714+0000 7efff3d40640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effec104af0 0x7effec197f20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7effdc00e9f0 tx=0x7effdc00eeb0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.715+0000 7efff3d40640 1 -- 172.21.15.17:0/1041274541 shutdown_connections 2024-03-24T03:00:22.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.715+0000 7efff3d40640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7effd80a2800 0x7effd80a4cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.715+0000 7efff3d40640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effec107420 0x7effec19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.715+0000 7efff3d40640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effec104af0 0x7effec197f20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.715+0000 7efff3d40640 1 --2- 172.21.15.17:0/1041274541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effec1021c0 0x7effec1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.715+0000 7efff3d40640 1 -- 172.21.15.17:0/1041274541 >> 172.21.15.17:0/1041274541 conn(0x7effec0fbbd0 msgr2=0x7effec102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:22.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.715+0000 7efff3d40640 1 -- 172.21.15.17:0/1041274541 shutdown_connections 2024-03-24T03:00:22.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.715+0000 7efff3d40640 1 -- 172.21.15.17:0/1041274541 wait complete. 2024-03-24T03:00:22.725 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T03:00:22.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.829+0000 7fda39d45640 1 -- 172.21.15.17:0/2292148780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda341021f0 msgr2=0x7fda34108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.829+0000 7fda39d45640 1 --2- 172.21.15.17:0/2292148780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda341021f0 0x7fda34108a80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009ca0 tx=0x7fda2c02c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.829+0000 7fda39d45640 1 -- 172.21.15.17:0/2292148780 shutdown_connections 2024-03-24T03:00:22.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.829+0000 7fda39d45640 1 --2- 172.21.15.17:0/2292148780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda341021f0 0x7fda34108a80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.829+0000 7fda39d45640 1 --2- 172.21.15.17:0/2292148780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda34101830 0x7fda34101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.829+0000 7fda39d45640 1 --2- 172.21.15.17:0/2292148780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda34100640 0x7fda34100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.829+0000 7fda39d45640 1 -- 172.21.15.17:0/2292148780 >> 172.21.15.17:0/2292148780 conn(0x7fda340fbbd0 msgr2=0x7fda340fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.829+0000 7fda39d45640 1 -- 172.21.15.17:0/2292148780 shutdown_connections 2024-03-24T03:00:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.829+0000 7fda39d45640 1 -- 172.21.15.17:0/2292148780 wait complete. 2024-03-24T03:00:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda39d45640 1 Processor -- start 2024-03-24T03:00:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda39d45640 1 -- start start 2024-03-24T03:00:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda39d45640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda34100640 0x7fda34196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda39d45640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda34101830 0x7fda34196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda39d45640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda341021f0 0x7fda3419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda39d45640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3410c250 con 0x7fda341021f0 2024-03-24T03:00:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda39d45640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3410c0d0 con 0x7fda34100640 2024-03-24T03:00:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda39d45640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3410c3d0 con 0x7fda34101830 2024-03-24T03:00:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda33fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda341021f0 0x7fda3419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:22.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda34100640 0x7fda34196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:22.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda33fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda341021f0 0x7fda3419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35506/0 (socket says 172.21.15.17:35506) 2024-03-24T03:00:22.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda33fff640 1 -- 172.21.15.17:0/2746293728 learned_addr learned my addr 172.21.15.17:0/2746293728 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:22.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda2bfff640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda34101830 0x7fda34196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:22.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda33fff640 1 -- 172.21.15.17:0/2746293728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda34101830 msgr2=0x7fda34196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda33fff640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda34101830 0x7fda34196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda33fff640 1 -- 172.21.15.17:0/2746293728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda34100640 msgr2=0x7fda34196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:22.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda33fff640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda34100640 0x7fda34196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:22.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.830+0000 7fda33fff640 1 -- 172.21.15.17:0/2746293728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3419e480 con 0x7fda341021f0 2024-03-24T03:00:22.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.831+0000 7fda33fff640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda341021f0 0x7fda3419de10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fda2c02c630 tx=0x7fda2c032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:22.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.831+0000 7fda317fa640 1 -- 172.21.15.17:0/2746293728 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c041020 con 0x7fda341021f0 2024-03-24T03:00:22.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.831+0000 7fda317fa640 1 -- 172.21.15.17:0/2746293728 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda2c034cd0 con 0x7fda341021f0 2024-03-24T03:00:22.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.831+0000 7fda39d45640 1 -- 172.21.15.17:0/2746293728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3419e710 con 0x7fda341021f0 2024-03-24T03:00:22.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.831+0000 7fda317fa640 1 -- 172.21.15.17:0/2746293728 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c039040 con 0x7fda341021f0 2024-03-24T03:00:22.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.831+0000 7fda39d45640 1 -- 172.21.15.17:0/2746293728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3419ec00 con 0x7fda341021f0 2024-03-24T03:00:22.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.832+0000 7fda39d45640 1 -- 172.21.15.17:0/2746293728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda34107960 con 0x7fda341021f0 2024-03-24T03:00:22.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.833+0000 7fda317fa640 1 -- 172.21.15.17:0/2746293728 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fda2c044020 con 0x7fda341021f0 2024-03-24T03:00:22.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.834+0000 7fda317fa640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda200a2800 0x7fda200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:22.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.834+0000 7fda337fe640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda200a2800 0x7fda200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:22.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.834+0000 7fda317fa640 1 -- 172.21.15.17:0/2746293728 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fda2c0f0fe0 con 0x7fda341021f0 2024-03-24T03:00:22.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.834+0000 7fda337fe640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda200a2800 0x7fda200a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda34101690 tx=0x7fda24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:22.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:22.837+0000 7fda317fa640 1 -- 172.21.15.17:0/2746293728 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fda2c04b020 con 0x7fda341021f0 2024-03-24T03:00:23.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.024+0000 7fda39d45640 1 -- 172.21.15.17:0/2746293728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda3402d240 con 0x7fda341021f0 2024-03-24T03:00:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.025+0000 7fda317fa640 1 -- 172.21.15.17:0/2746293728 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24514 (secure 0 0 0) 0x7fda2c0bdde0 con 0x7fda341021f0 2024-03-24T03:00:23.025 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:23.025 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":91,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T02:59:30.964668+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T03:00:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.028+0000 7fda39d45640 1 -- 172.21.15.17:0/2746293728 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda200a2800 msgr2=0x7fda200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.028+0000 7fda39d45640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda200a2800 0x7fda200a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda34101690 tx=0x7fda24005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.028+0000 7fda39d45640 1 -- 172.21.15.17:0/2746293728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda341021f0 msgr2=0x7fda3419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.028+0000 7fda39d45640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda341021f0 0x7fda3419de10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fda2c02c630 tx=0x7fda2c032ea0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.028+0000 7fda39d45640 1 -- 172.21.15.17:0/2746293728 shutdown_connections 2024-03-24T03:00:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.028+0000 7fda39d45640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fda200a2800 0x7fda200a4cc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.028+0000 7fda39d45640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda341021f0 0x7fda3419de10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.028+0000 7fda39d45640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda34101830 0x7fda34196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.028+0000 7fda39d45640 1 --2- 172.21.15.17:0/2746293728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda34100640 0x7fda34196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.028+0000 7fda39d45640 1 -- 172.21.15.17:0/2746293728 >> 172.21.15.17:0/2746293728 conn(0x7fda340fbbd0 msgr2=0x7fda34104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.028+0000 7fda39d45640 1 -- 172.21.15.17:0/2746293728 shutdown_connections 2024-03-24T03:00:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.028+0000 7fda39d45640 1 -- 172.21.15.17:0/2746293728 wait complete. 2024-03-24T03:00:23.037 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:22 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3039524379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:00:23.037 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:22 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2229748220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:00:23.037 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:22 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1041274541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:23.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T03:00:23.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:22 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3039524379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:00:23.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:22 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2229748220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:00:23.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:22 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1041274541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:23.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.142+0000 7f5a29d74640 1 -- 172.21.15.17:0/2513955985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a24102170 msgr2=0x7f5a24108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.142+0000 7f5a29d74640 1 --2- 172.21.15.17:0/2513955985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a24102170 0x7f5a24108a00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009900 tx=0x7f5a1c02bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:23.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.142+0000 7f5a29d74640 1 -- 172.21.15.17:0/2513955985 shutdown_connections 2024-03-24T03:00:23.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.142+0000 7f5a29d74640 1 --2- 172.21.15.17:0/2513955985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a24102170 0x7f5a24108a00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.142+0000 7f5a29d74640 1 --2- 172.21.15.17:0/2513955985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a241017b0 0x7f5a24101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.142+0000 7f5a29d74640 1 --2- 172.21.15.17:0/2513955985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a241005c0 0x7f5a241009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.142+0000 7f5a29d74640 1 -- 172.21.15.17:0/2513955985 >> 172.21.15.17:0/2513955985 conn(0x7f5a240fbb30 msgr2=0x7f5a240fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.142+0000 7f5a29d74640 1 -- 172.21.15.17:0/2513955985 shutdown_connections 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.142+0000 7f5a29d74640 1 -- 172.21.15.17:0/2513955985 wait complete. 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a29d74640 1 Processor -- start 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a29d74640 1 -- start start 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a29d74640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a241005c0 0x7f5a24069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a29d74640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a241017b0 0x7f5a2419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a29d74640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a24102170 0x7f5a2419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a29d74640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2410c270 con 0x7f5a24102170 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a29d74640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2410c0f0 con 0x7f5a241005c0 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a29d74640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2410c3f0 con 0x7f5a241017b0 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a23fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a24102170 0x7f5a2419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a23fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a24102170 0x7f5a2419ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35534/0 (socket says 172.21.15.17:35534) 2024-03-24T03:00:23.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a23fff640 1 -- 172.21.15.17:0/3642083499 learned_addr learned my addr 172.21.15.17:0/3642083499 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a237fe640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a241005c0 0x7f5a24069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.143+0000 7f5a22ffd640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a241017b0 0x7f5a2419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.144+0000 7f5a23fff640 1 -- 172.21.15.17:0/3642083499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a241017b0 msgr2=0x7f5a2419c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.144+0000 7f5a23fff640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a241017b0 0x7f5a2419c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.144+0000 7f5a23fff640 1 -- 172.21.15.17:0/3642083499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a241005c0 msgr2=0x7f5a24069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.144+0000 7f5a23fff640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a241005c0 0x7f5a24069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.144+0000 7f5a23fff640 1 -- 172.21.15.17:0/3642083499 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2419f5b0 con 0x7f5a24102170 2024-03-24T03:00:23.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.144+0000 7f5a23fff640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a24102170 0x7f5a2419ecf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c02c8e0 tx=0x7f5a1c02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.144+0000 7f5a20ff9640 1 -- 172.21.15.17:0/3642083499 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c043020 con 0x7f5a24102170 2024-03-24T03:00:23.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.144+0000 7f5a20ff9640 1 -- 172.21.15.17:0/3642083499 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a1c0054e0 con 0x7f5a24102170 2024-03-24T03:00:23.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.144+0000 7f5a20ff9640 1 -- 172.21.15.17:0/3642083499 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c003d00 con 0x7f5a24102170 2024-03-24T03:00:23.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.144+0000 7f5a29d74640 1 -- 172.21.15.17:0/3642083499 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2419f840 con 0x7f5a24102170 2024-03-24T03:00:23.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.144+0000 7f5a29d74640 1 -- 172.21.15.17:0/3642083499 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2419fd90 con 0x7f5a24102170 2024-03-24T03:00:23.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.145+0000 7f5a29d74640 1 -- 172.21.15.17:0/3642083499 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a241078e0 con 0x7f5a24102170 2024-03-24T03:00:23.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.146+0000 7f5a20ff9640 1 -- 172.21.15.17:0/3642083499 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5a1c046020 con 0x7f5a24102170 2024-03-24T03:00:23.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.147+0000 7f5a20ff9640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a100a2800 0x7f5a100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.147+0000 7f5a237fe640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a100a2800 0x7f5a100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.147+0000 7f5a20ff9640 1 -- 172.21.15.17:0/3642083499 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f5a1c0f0e40 con 0x7f5a24102170 2024-03-24T03:00:23.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.147+0000 7f5a237fe640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a100a2800 0x7f5a100a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5a24101610 tx=0x7f5a14009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.151+0000 7f5a20ff9640 1 -- 172.21.15.17:0/3642083499 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a1c04d020 con 0x7f5a24102170 2024-03-24T03:00:23.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:22 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3039524379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:00:23.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:22 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2229748220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:00:23.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:22 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1041274541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:23.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.373+0000 7f5a29d74640 1 -- 172.21.15.17:0/3642083499 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a240622a0 con 0x7f5a24102170 2024-03-24T03:00:23.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.374+0000 7f5a20ff9640 1 -- 172.21.15.17:0/3642083499 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11452 (secure 0 0 0) 0x7f5a1c0bdbe0 con 0x7f5a24102170 2024-03-24T03:00:23.373 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:23.374 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T02:58:36.476051+0000","modified":"2024-03-24T02:58:59.074424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24529,"mds_1":24521,"mds_2":24511,"mds_3":24493,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6841/1592961709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":1592961709},{"type":"v1","addr":"172.21.15.17:6841","nonce":1592961709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6839/2102503456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2102503456},{"type":"v1","addr":"172.21.15.19:6839","nonce":2102503456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24511,24529,24485,24493,24521]},"id":1}]} 2024-03-24T03:00:23.374 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 28 2024-03-24T03:00:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.376+0000 7f5a29d74640 1 -- 172.21.15.17:0/3642083499 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a100a2800 msgr2=0x7f5a100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.376+0000 7f5a29d74640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a100a2800 0x7f5a100a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5a24101610 tx=0x7f5a14009500 comp rx=0 tx=0).stop 2024-03-24T03:00:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.376+0000 7f5a29d74640 1 -- 172.21.15.17:0/3642083499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a24102170 msgr2=0x7f5a2419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.376+0000 7f5a29d74640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a24102170 0x7f5a2419ecf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c02c8e0 tx=0x7f5a1c02c640 comp rx=0 tx=0).stop 2024-03-24T03:00:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.376+0000 7f5a29d74640 1 -- 172.21.15.17:0/3642083499 shutdown_connections 2024-03-24T03:00:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.376+0000 7f5a29d74640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f5a100a2800 0x7f5a100a4cc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.376+0000 7f5a29d74640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a24102170 0x7f5a2419ecf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.376+0000 7f5a29d74640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a241017b0 0x7f5a2419c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.376+0000 7f5a29d74640 1 --2- 172.21.15.17:0/3642083499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a241005c0 0x7f5a24069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.376+0000 7f5a29d74640 1 -- 172.21.15.17:0/3642083499 >> 172.21.15.17:0/3642083499 conn(0x7f5a240fbb30 msgr2=0x7f5a24104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.376+0000 7f5a29d74640 1 -- 172.21.15.17:0/3642083499 shutdown_connections 2024-03-24T03:00:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.376+0000 7f5a29d74640 1 -- 172.21.15.17:0/3642083499 wait complete. 2024-03-24T03:00:23.387 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-24T02:58:36.476051+0000', 'modified': '2024-03-24T02:58:59.074424+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24529, 'mds_1': 24521, 'mds_2': 24511, 'mds_3': 24493, 'mds_4': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.17:6839/2895019504', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6838', 'nonce': 2895019504}, {'type': 'v1', 'addr': '172.21.15.17:6839', 'nonce': 2895019504}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.17:6841/1592961709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6840', 'nonce': 1592961709}, {'type': 'v1', 'addr': '172.21.15.17:6841', 'nonce': 1592961709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24419': {'gid': 24419, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6833/3469034636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 3469034636}, {'type': 'v1', 'addr': '172.21.15.19:6833', 'nonce': 3469034636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6833/920982807', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 920982807}, {'type': 'v1', 'addr': '172.21.15.188:6833', 'nonce': 920982807}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/1025006466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 1025006466}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 1025006466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.19:6837/2540260543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6836', 'nonce': 2540260543}, {'type': 'v1', 'addr': '172.21.15.19:6837', 'nonce': 2540260543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.188:6835/3649219818', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 3649219818}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 3649219818}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.188:6837/3512350624', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 3512350624}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 3512350624}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.19:6839/2102503456', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 2102503456}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 2102503456}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.188:6839/81409664', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 81409664}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 81409664}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24529, 'qdb_cluster': [24511, 24529, 24485, 24493, 24521]} 2024-03-24T03:00:23.387 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-24T03:00:23.387 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-24T03:00:23.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.491+0000 7f09bef1b640 1 -- 172.21.15.17:0/736795518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09b80ff970 msgr2=0x7f09b80ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.491+0000 7f09bef1b640 1 --2- 172.21.15.17:0/736795518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09b80ff970 0x7f09b80ffd90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009cd0 tx=0x7f09b002c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.491+0000 7f09bef1b640 1 -- 172.21.15.17:0/736795518 shutdown_connections 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.491+0000 7f09bef1b640 1 --2- 172.21.15.17:0/736795518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09b8103300 0x7f09b8109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.491+0000 7f09bef1b640 1 --2- 172.21.15.17:0/736795518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b8102940 0x7f09b8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.491+0000 7f09bef1b640 1 --2- 172.21.15.17:0/736795518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09b80ff970 0x7f09b80ffd90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.491+0000 7f09bef1b640 1 -- 172.21.15.17:0/736795518 >> 172.21.15.17:0/736795518 conn(0x7f09b80fb2c0 msgr2=0x7f09b80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.491+0000 7f09bef1b640 1 -- 172.21.15.17:0/736795518 shutdown_connections 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.491+0000 7f09bef1b640 1 -- 172.21.15.17:0/736795518 wait complete. 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bef1b640 1 Processor -- start 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bef1b640 1 -- start start 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bef1b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b80ff970 0x7f09b81022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bef1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09b8102940 0x7f09b81008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bef1b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09b8103300 0x7f09b8100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bef1b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b810d350 con 0x7f09b8103300 2024-03-24T03:00:23.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bef1b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b810d1d0 con 0x7f09b8102940 2024-03-24T03:00:23.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bef1b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b810d4d0 con 0x7f09b80ff970 2024-03-24T03:00:23.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bd4b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09b8103300 0x7f09b8100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bccb7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b80ff970 0x7f09b81022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bd4b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09b8103300 0x7f09b8100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35554/0 (socket says 172.21.15.17:35554) 2024-03-24T03:00:23.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09b8102940 0x7f09b81008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bd4b8640 1 -- 172.21.15.17:0/984809944 learned_addr learned my addr 172.21.15.17:0/984809944 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.492+0000 7f09bd4b8640 1 -- 172.21.15.17:0/984809944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b80ff970 msgr2=0x7f09b81022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.493+0000 7f09bd4b8640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b80ff970 0x7f09b81022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.493+0000 7f09bd4b8640 1 -- 172.21.15.17:0/984809944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09b8102940 msgr2=0x7f09b81008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.493+0000 7f09bd4b8640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09b8102940 0x7f09b81008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.493+0000 7f09bd4b8640 1 -- 172.21.15.17:0/984809944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b81016c0 con 0x7f09b8103300 2024-03-24T03:00:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.493+0000 7f09bccb7640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b80ff970 0x7f09b81022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.493+0000 7f09b7fff640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09b8102940 0x7f09b81008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.493+0000 7f09bd4b8640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09b8103300 0x7f09b8100e30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f09a000e970 tx=0x7f09a000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.493+0000 7f09b5ffb640 1 -- 172.21.15.17:0/984809944 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a000ccc0 con 0x7f09b8103300 2024-03-24T03:00:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.493+0000 7f09bef1b640 1 -- 172.21.15.17:0/984809944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b81a1b60 con 0x7f09b8103300 2024-03-24T03:00:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.493+0000 7f09b5ffb640 1 -- 172.21.15.17:0/984809944 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09a000ce60 con 0x7f09b8103300 2024-03-24T03:00:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.493+0000 7f09b5ffb640 1 -- 172.21.15.17:0/984809944 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0010490 con 0x7f09b8103300 2024-03-24T03:00:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.493+0000 7f09bef1b640 1 -- 172.21.15.17:0/984809944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b81a20b0 con 0x7f09b8103300 2024-03-24T03:00:23.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.495+0000 7f09b5ffb640 1 -- 172.21.15.17:0/984809944 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f09a0005020 con 0x7f09b8103300 2024-03-24T03:00:23.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.495+0000 7f09c1be2740 1 Processor -- start 2024-03-24T03:00:23.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.495+0000 7f09c1be2740 1 -- start start 2024-03-24T03:00:23.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09c1be2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afa740 0x55fe28afab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09c1be2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 0x55fe28affc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09c1be2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28b002a0 0x55fe28b02730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09c1be2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe28a5fc20 con 0x55fe28b002a0 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09c1be2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe28a08a40 con 0x55fe28afb130 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09c1be2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe28a5bcc0 con 0x55fe28afa740 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09b5ffb640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f09a40a2800 0x7f09a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09b7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28b002a0 0x55fe28b02730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09bccb7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 0x55fe28affc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09bccb7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 0x55fe28affc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:44580/0 (socket says 172.21.15.17:44580) 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09bccb7640 1 -- 172.21.15.17:0/2650987954 learned_addr learned my addr 172.21.15.17:0/2650987954 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09bd4b8640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f09a40a2800 0x7f09a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09b7fff640 1 --2- 172.21.15.17:0/2650987954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afa740 0x55fe28afab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09b5ffb640 1 -- 172.21.15.17:0/984809944 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f09a00cf2b0 con 0x7f09b8103300 2024-03-24T03:00:23.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09bccb7640 1 -- 172.21.15.17:0/2650987954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afa740 msgr2=0x55fe28afab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09bccb7640 1 --2- 172.21.15.17:0/2650987954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afa740 0x55fe28afab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09bccb7640 1 -- 172.21.15.17:0/2650987954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28b002a0 msgr2=0x55fe28b02730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09bccb7640 1 --2- 172.21.15.17:0/2650987954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28b002a0 0x55fe28b02730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09bccb7640 1 -- 172.21.15.17:0/2650987954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe28a2af90 con 0x55fe28afb130 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09bd4b8640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f09a40a2800 0x7f09a40a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f09a009b850 tx=0x7f09a0086040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.496+0000 7f09bccb7640 1 --2- 172.21.15.17:0/2650987954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 0x55fe28affc30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f09b0005e30 tx=0x7f09b0005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c55bbacefd1f47c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.497+0000 7f09aeffd640 1 -- 172.21.15.17:0/2650987954 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0043070 con 0x55fe28afb130 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.497+0000 7f09aeffd640 1 -- 172.21.15.17:0/2650987954 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09b0003970 con 0x55fe28afb130 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.497+0000 7f09aeffd640 1 -- 172.21.15.17:0/2650987954 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0039040 con 0x55fe28afb130 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.497+0000 7f09c1be2740 1 -- 172.21.15.17:0/2650987954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 msgr2=0x55fe28affc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.497+0000 7f09c1be2740 1 --2- 172.21.15.17:0/2650987954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 0x55fe28affc30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f09b0005e30 tx=0x7f09b0005ef0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.497+0000 7f09c1be2740 1 -- 172.21.15.17:0/2650987954 shutdown_connections 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.497+0000 7f09c1be2740 1 --2- 172.21.15.17:0/2650987954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28b002a0 0x55fe28b02730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.497+0000 7f09c1be2740 1 --2- 172.21.15.17:0/2650987954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 0x55fe28affc30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.497+0000 7f09c1be2740 1 --2- 172.21.15.17:0/2650987954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afa740 0x55fe28afab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.497+0000 7f09c1be2740 1 -- 172.21.15.17:0/2650987954 >> 172.21.15.17:0/2650987954 conn(0x55fe28ac2a60 msgr2=0x55fe28af60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.497+0000 7f09c1be2740 1 -- 172.21.15.17:0/2650987954 shutdown_connections 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.497+0000 7f09c1be2740 1 -- 172.21.15.17:0/2650987954 wait complete. 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09c1be2740 1 -- start start 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09c1be2740 1 Processor -- start 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09c1be2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 0x55fe28b04c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09c1be2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afb130 0x55fe28b05140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09c1be2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28b002a0 0x55fe28b0c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09c1be2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe28b031f0 con 0x55fe28afa740 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09c1be2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe28a1be90 con 0x55fe28b002a0 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09c1be2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe28b03370 con 0x55fe28afb130 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09c1be2740 10 client.0 _ll_register_callbacks cb 0x55fe28773ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09b7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 0x55fe28b04c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09b7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 0x55fe28b04c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35580/0 (socket says 172.21.15.17:35580) 2024-03-24T03:00:23.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09b7fff640 1 -- 172.21.15.17:0/3973931290 learned_addr learned my addr 172.21.15.17:0/3973931290 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09bccb7640 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afb130 0x55fe28b05140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09b7fff640 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28b002a0 0x55fe28b0c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09bccb7640 1 -- 172.21.15.17:0/3973931290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28b002a0 msgr2=0x55fe28b0c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09bccb7640 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28b002a0 0x55fe28b0c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09bccb7640 1 -- 172.21.15.17:0/3973931290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 msgr2=0x55fe28b04c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09bccb7640 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 0x55fe28b04c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09bccb7640 1 -- 172.21.15.17:0/3973931290 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe28a289d0 con 0x55fe28afb130 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.498+0000 7f09b7fff640 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 0x55fe28b04c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f09bccb7640 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afb130 0x55fe28b05140 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55fe28af7000 tx=0x7f09b0003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f1201f7923b26e server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f0997fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f0997fff640 1 -- 172.21.15.17:0/3973931290 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b004c030 con 0x55fe28afb130 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f0997fff640 1 -- 172.21.15.17:0/3973931290 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f09b0032690 con 0x55fe28afb130 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f09c1be2740 1 -- 172.21.15.17:0/3973931290 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09b0003970 con 0x55fe28afb130 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f0997fff640 1 -- 172.21.15.17:0/3973931290 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09b0039070 con 0x55fe28afb130 2024-03-24T03:00:23.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f09c1be2740 10 client.25120 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f09c1be2740 1 -- 172.21.15.17:0/3973931290 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fe287742d0 con 0x55fe28afb130 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f0997fff640 1 -- 172.21.15.17:0/3973931290 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09b0003970 con 0x55fe28afb130 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f0997fff640 1 -- 172.21.15.17:0/3973931290 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f09b00489a0 con 0x55fe28afb130 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f09c1be2740 10 client.25120 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f09c1be2740 10 client.25120 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f09c1be2740 20 client.25120 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f09c1be2740 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x55fe28b102e0 0x55fe28b15320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f09c1be2740 4 client.25120 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.499+0000 7f09c1be2740 1 -- 172.21.15.17:0/3973931290 --> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe28a6d160 con 0x55fe28b102e0 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.500+0000 7f09b7fff640 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x55fe28b102e0 0x55fe28b15320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.500+0000 7f0997fff640 10 client.25120 ms_handle_connect on v2:172.21.15.19:6836/2540260543 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.500+0000 7f09b7fff640 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x55fe28b102e0 0x55fe28b15320 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b909b9f676fbafbf server_cookie=fe8cfeabbb1778f6 in_seq=0 out_seq=0 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f0997fff640 1 -- 172.21.15.17:0/3973931290 <== mds.4 v2:172.21.15.19:6836/2540260543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fe28a6d160 con 0x55fe28b102e0 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f0997fff640 10 client.25120 handle_command_reply: tid=0 2024-03-24T03:00:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f09c1be2740 1 client.25120 shutdown 2024-03-24T03:00:23.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f09c1be2740 1 -- 172.21.15.17:0/3973931290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afb130 msgr2=0x55fe28b05140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f09c1be2740 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afb130 0x55fe28b05140 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55fe28af7000 tx=0x7f09b0003f80 comp rx=0 tx=0).stop 2024-03-24T03:00:23.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f09c1be2740 1 -- 172.21.15.17:0/3973931290 shutdown_connections 2024-03-24T03:00:23.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f09c1be2740 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x55fe28b102e0 0x55fe28b15320 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f09c1be2740 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28b002a0 0x55fe28b0c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f09c1be2740 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afb130 0x55fe28b05140 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f09c1be2740 1 --2- 172.21.15.17:0/3973931290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 0x55fe28b04c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f0997fff640 0 client.25120 ms_handle_reset on v2:172.21.15.19:6836/2540260543 2024-03-24T03:00:23.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f09c1be2740 1 -- 172.21.15.17:0/3973931290 >> 172.21.15.17:0/3973931290 conn(0x55fe28ac2a60 msgr2=0x55fe28af52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f09c1be2740 1 -- 172.21.15.17:0/3973931290 shutdown_connections 2024-03-24T03:00:23.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f09c1be2740 1 -- 172.21.15.17:0/3973931290 wait complete. 2024-03-24T03:00:23.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.501+0000 7f09c1be2740 20 client.25120 trim_cache size 0 max 16384 2024-03-24T03:00:23.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09c1be2740 1 Processor -- start 2024-03-24T03:00:23.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09c1be2740 1 -- start start 2024-03-24T03:00:23.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09c1be2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 0x55fe28b0c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09c1be2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 0x55fe28b0c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09c1be2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28b002a0 0x55fe28b0cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09c1be2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe28b106c0 con 0x55fe28afa740 2024-03-24T03:00:23.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09c1be2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe28b10d20 con 0x55fe28afb130 2024-03-24T03:00:23.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09c1be2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe28b11380 con 0x55fe28b002a0 2024-03-24T03:00:23.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09b7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 0x55fe28b0c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09b7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 0x55fe28b0c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35586/0 (socket says 172.21.15.17:35586) 2024-03-24T03:00:23.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09b7fff640 1 -- 172.21.15.17:0/1668465945 learned_addr learned my addr 172.21.15.17:0/1668465945 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09bccb7640 1 --2- 172.21.15.17:0/1668465945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 0x55fe28b0c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09b7fff640 1 --2- 172.21.15.17:0/1668465945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28b002a0 0x55fe28b0cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09bccb7640 1 -- 172.21.15.17:0/1668465945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28b002a0 msgr2=0x55fe28b0cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09bccb7640 1 --2- 172.21.15.17:0/1668465945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28b002a0 0x55fe28b0cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09bccb7640 1 -- 172.21.15.17:0/1668465945 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 msgr2=0x55fe28b0c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09bccb7640 1 --2- 172.21.15.17:0/1668465945 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 0x55fe28b0c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09bccb7640 1 -- 172.21.15.17:0/1668465945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe2883e6b0 con 0x55fe28afb130 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09b7fff640 1 --2- 172.21.15.17:0/1668465945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28b002a0 0x55fe28b0cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09bccb7640 1 --2- 172.21.15.17:0/1668465945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 0x55fe28b0c140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f09b0048d40 tx=0x7f09b0000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5471929a86dacd60 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.518+0000 7f09ae7fc640 1 -- 172.21.15.17:0/1668465945 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b004c020 con 0x55fe28afb130 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09ae7fc640 1 -- 172.21.15.17:0/1668465945 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09b0003970 con 0x55fe28afb130 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09ae7fc640 1 -- 172.21.15.17:0/1668465945 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0043040 con 0x55fe28afb130 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09c1be2740 1 -- 172.21.15.17:0/1668465945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 msgr2=0x55fe28b0c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09c1be2740 1 --2- 172.21.15.17:0/1668465945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 0x55fe28b0c140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f09b0048d40 tx=0x7f09b0000f40 comp rx=0 tx=0).stop 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09c1be2740 1 -- 172.21.15.17:0/1668465945 shutdown_connections 2024-03-24T03:00:23.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09c1be2740 1 --2- 172.21.15.17:0/1668465945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28b002a0 0x55fe28b0cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09c1be2740 1 --2- 172.21.15.17:0/1668465945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afb130 0x55fe28b0c140 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09c1be2740 1 --2- 172.21.15.17:0/1668465945 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28afa740 0x55fe28b0c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09c1be2740 1 -- 172.21.15.17:0/1668465945 >> 172.21.15.17:0/1668465945 conn(0x55fe28ac2a60 msgr2=0x55fe28b04c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09c1be2740 1 -- 172.21.15.17:0/1668465945 shutdown_connections 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09c1be2740 1 -- 172.21.15.17:0/1668465945 wait complete. 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09c1be2740 1 -- start start 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.519+0000 7f09c1be2740 1 Processor -- start 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09c1be2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afa740 0x55fe28b07d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09c1be2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afb130 0x55fe28b08250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09c1be2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28b002a0 0x55fe28b0b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09c1be2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe28b12040 con 0x55fe28b002a0 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09c1be2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe28b119e0 con 0x55fe28afa740 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09c1be2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe28b126a0 con 0x55fe28afb130 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09c1be2740 10 client.0 _ll_register_callbacks cb 0x7f09a8009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:23.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afa740 0x55fe28b07d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afa740 0x55fe28b07d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:44592/0 (socket says 172.21.15.17:44592) 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09b7fff640 1 -- 172.21.15.17:0/3455485357 learned_addr learned my addr 172.21.15.17:0/3455485357 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09bccb7640 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afb130 0x55fe28b08250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09b7fff640 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28b002a0 0x55fe28b0b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09b7fff640 1 -- 172.21.15.17:0/3455485357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afb130 msgr2=0x55fe28b08250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09b7fff640 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afb130 0x55fe28b08250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09b7fff640 1 -- 172.21.15.17:0/3455485357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28b002a0 msgr2=0x55fe28b0b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09b7fff640 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28b002a0 0x55fe28b0b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09b7fff640 1 -- 172.21.15.17:0/3455485357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe28b0b860 con 0x55fe28afa740 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f09b7fff640 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afa740 0x55fe28b07d10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09a8004220 tx=0x7f09a8027040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=424412b37d408cf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.520+0000 7f0997fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f0997fff640 1 -- 172.21.15.17:0/3455485357 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a8011110 con 0x55fe28afa740 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f09c1be2740 1 -- 172.21.15.17:0/3455485357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09b0003970 con 0x55fe28afa740 2024-03-24T03:00:23.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f0997fff640 1 -- 172.21.15.17:0/3455485357 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f09a801a040 con 0x55fe28afa740 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f0997fff640 1 -- 172.21.15.17:0/3455485357 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09a803b050 con 0x55fe28afa740 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f09c1be2740 10 client.25013 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f09c1be2740 1 -- 172.21.15.17:0/3455485357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fe28b6c9e0 con 0x55fe28afa740 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f0997fff640 1 -- 172.21.15.17:0/3455485357 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09b0003970 con 0x55fe28afa740 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f0997fff640 1 -- 172.21.15.17:0/3455485357 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f09a8020040 con 0x55fe28afa740 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f09c1be2740 10 client.25013 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f09c1be2740 10 client.25013 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f09c1be2740 20 client.25013 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f09c1be2740 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x55fe28b71040 0x55fe28b6cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f09c1be2740 4 client.25013 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f09c1be2740 1 -- 172.21.15.17:0/3455485357 --> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fe28a674b0 con 0x55fe28b71040 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.521+0000 7f09bccb7640 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x55fe28b71040 0x55fe28b6cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09bccb7640 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x55fe28b71040 0x55fe28b6cdb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8fbc2cf9d76b4d3f server_cookie=bb417ed624d93ac6 in_seq=0 out_seq=0 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f0997fff640 10 client.25013 ms_handle_connect on v2:172.21.15.19:6836/2540260543 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f0997fff640 1 -- 172.21.15.17:0/3455485357 <== mds.4 v2:172.21.15.19:6836/2540260543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55fe28a674b0 con 0x55fe28b71040 2024-03-24T03:00:23.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f0997fff640 10 client.25013 handle_command_reply: tid=0 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09c1be2740 1 client.25013 shutdown 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09c1be2740 1 -- 172.21.15.17:0/3455485357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afa740 msgr2=0x55fe28b07d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09c1be2740 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afa740 0x55fe28b07d10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09a8004220 tx=0x7f09a8027040 comp rx=0 tx=0).stop 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09c1be2740 1 -- 172.21.15.17:0/3455485357 shutdown_connections 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09c1be2740 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x55fe28b71040 0x55fe28b6cdb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09c1be2740 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe28b002a0 0x55fe28b0b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f0997fff640 0 client.25013 ms_handle_reset on v2:172.21.15.19:6836/2540260543 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09c1be2740 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe28afb130 0x55fe28b08250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09c1be2740 1 --2- 172.21.15.17:0/3455485357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe28afa740 0x55fe28b07d10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09c1be2740 1 -- 172.21.15.17:0/3455485357 >> 172.21.15.17:0/3455485357 conn(0x55fe28ac2a60 msgr2=0x55fe28b0c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09c1be2740 1 -- 172.21.15.17:0/3455485357 shutdown_connections 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09c1be2740 1 -- 172.21.15.17:0/3455485357 wait complete. 2024-03-24T03:00:23.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.522+0000 7f09c1be2740 20 client.25013 trim_cache size 0 max 16384 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 4, 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24485, 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 85.450708574000004, 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 28, 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 91, 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 123.961617477 2024-03-24T03:00:23.529 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:00:23.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.523+0000 7f09bef1b640 1 -- 172.21.15.17:0/984809944 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f09a40a2800 msgr2=0x7f09a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.523+0000 7f09bef1b640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f09a40a2800 0x7f09a40a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f09a009b850 tx=0x7f09a0086040 comp rx=0 tx=0).stop 2024-03-24T03:00:23.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.523+0000 7f09bef1b640 1 -- 172.21.15.17:0/984809944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09b8103300 msgr2=0x7f09b8100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.523+0000 7f09bef1b640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09b8103300 0x7f09b8100e30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f09a000e970 tx=0x7f09a000ed70 comp rx=0 tx=0).stop 2024-03-24T03:00:23.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.523+0000 7f09bef1b640 1 -- 172.21.15.17:0/984809944 shutdown_connections 2024-03-24T03:00:23.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.523+0000 7f09bef1b640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f09a40a2800 0x7f09a40a4cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.523+0000 7f09bef1b640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09b8103300 0x7f09b8100e30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.523+0000 7f09bef1b640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09b8102940 0x7f09b81008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.523+0000 7f09bef1b640 1 --2- 172.21.15.17:0/984809944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b80ff970 0x7f09b81022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.523+0000 7f09bef1b640 1 -- 172.21.15.17:0/984809944 >> 172.21.15.17:0/984809944 conn(0x7f09b80fb2c0 msgr2=0x7f09b8105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.523+0000 7f09bef1b640 1 -- 172.21.15.17:0/984809944 shutdown_connections 2024-03-24T03:00:23.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.523+0000 7f09bef1b640 1 -- 172.21.15.17:0/984809944 wait complete. 2024-03-24T03:00:23.533 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.637+0000 7f953dff3640 1 -- 172.21.15.17:0/4069392560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9538102150 msgr2=0x7f95381089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.637+0000 7f953dff3640 1 --2- 172.21.15.17:0/4069392560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9538102150 0x7f95381089e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9530009be0 tx=0x7f953002bef0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 -- 172.21.15.17:0/4069392560 shutdown_connections 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 --2- 172.21.15.17:0/4069392560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9538102150 0x7f95381089e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 --2- 172.21.15.17:0/4069392560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9538101790 0x7f9538101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 --2- 172.21.15.17:0/4069392560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95381005a0 0x7f95381009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 -- 172.21.15.17:0/4069392560 >> 172.21.15.17:0/4069392560 conn(0x7f95380fbb30 msgr2=0x7f95380fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 -- 172.21.15.17:0/4069392560 shutdown_connections 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 -- 172.21.15.17:0/4069392560 wait complete. 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 Processor -- start 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 -- start start 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95381005a0 0x7f9538196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9538101790 0x7f9538196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9538102150 0x7f953819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953810c230 con 0x7f9538102150 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953810c0b0 con 0x7f9538101790 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.638+0000 7f953dff3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953810c3b0 con 0x7f95381005a0 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9537fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9538102150 0x7f953819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9537fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9538102150 0x7f953819de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35626/0 (socket says 172.21.15.17:35626) 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9537fff640 1 -- 172.21.15.17:0/2989358456 learned_addr learned my addr 172.21.15.17:0/2989358456 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f95377fe640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95381005a0 0x7f9538196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9536ffd640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9538101790 0x7f9538196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9537fff640 1 -- 172.21.15.17:0/2989358456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95381005a0 msgr2=0x7f9538196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9537fff640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95381005a0 0x7f9538196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9537fff640 1 -- 172.21.15.17:0/2989358456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9538101790 msgr2=0x7f9538196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9537fff640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9538101790 0x7f9538196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9537fff640 1 -- 172.21.15.17:0/2989358456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953819e4a0 con 0x7f9538102150 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9537fff640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9538102150 0x7f953819de30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9530036040 tx=0x7f95300337b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9534ff9640 1 -- 172.21.15.17:0/2989358456 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530043020 con 0x7f9538102150 2024-03-24T03:00:23.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f953dff3640 1 -- 172.21.15.17:0/2989358456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953819e730 con 0x7f9538102150 2024-03-24T03:00:23.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9534ff9640 1 -- 172.21.15.17:0/2989358456 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f953002ca60 con 0x7f9538102150 2024-03-24T03:00:23.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f9534ff9640 1 -- 172.21.15.17:0/2989358456 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530034700 con 0x7f9538102150 2024-03-24T03:00:23.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.639+0000 7f953dff3640 1 -- 172.21.15.17:0/2989358456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953819ec80 con 0x7f9538102150 2024-03-24T03:00:23.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.641+0000 7f9534ff9640 1 -- 172.21.15.17:0/2989358456 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9530046020 con 0x7f9538102150 2024-03-24T03:00:23.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9540bda740 1 Processor -- start 2024-03-24T03:00:23.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9534ff9640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f95200a2800 0x7f95200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9540bda740 1 -- start start 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9536ffd640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f95200a2800 0x7f95200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9540bda740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca1740 0x559173ca1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9534ff9640 1 -- 172.21.15.17:0/2989358456 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f95300f12e0 con 0x7f9538102150 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9540bda740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173ca6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9540bda740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca72a0 0x559173ca9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9540bda740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559173c06c20 con 0x559173ca2130 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9540bda740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559173bafa40 con 0x559173ca1740 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9540bda740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559173c02cc0 con 0x559173ca72a0 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9536ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173ca6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f95377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca1740 0x559173ca1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9536ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173ca6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35642/0 (socket says 172.21.15.17:35642) 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9536ffd640 1 -- 172.21.15.17:0/2848270372 learned_addr learned my addr 172.21.15.17:0/2848270372 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9537fff640 1 --2- 172.21.15.17:0/2848270372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca72a0 0x559173ca9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9536ffd640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f95200a2800 0x7f95200a4cc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f952c013570 tx=0x7f952c009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9537fff640 1 -- 172.21.15.17:0/2848270372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca1740 msgr2=0x559173ca1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9537fff640 1 --2- 172.21.15.17:0/2848270372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca1740 0x559173ca1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9537fff640 1 -- 172.21.15.17:0/2848270372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 msgr2=0x559173ca6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9537fff640 1 --2- 172.21.15.17:0/2848270372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173ca6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f9537fff640 1 -- 172.21.15.17:0/2848270372 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559173bd1f90 con 0x559173ca72a0 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.643+0000 7f9536ffd640 1 --2- 172.21.15.17:0/2848270372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173ca6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.642+0000 7f95377fe640 1 --2- 172.21.15.17:0/2848270372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca1740 0x559173ca1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.643+0000 7f9537fff640 1 --2- 172.21.15.17:0/2848270372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca72a0 0x559173ca9730 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f95300eeba0 tx=0x7f953002bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=363188b803486813 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.643+0000 7f951dffb640 1 -- 172.21.15.17:0/2848270372 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95300be0f0 con 0x559173ca72a0 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.643+0000 7f951dffb640 1 -- 172.21.15.17:0/2848270372 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95300bd9f0 con 0x559173ca72a0 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.643+0000 7f951dffb640 1 -- 172.21.15.17:0/2848270372 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95300ba060 con 0x559173ca72a0 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.643+0000 7f9540bda740 1 -- 172.21.15.17:0/2848270372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca72a0 msgr2=0x559173ca9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.643+0000 7f9540bda740 1 --2- 172.21.15.17:0/2848270372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca72a0 0x559173ca9730 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f95300eeba0 tx=0x7f953002bef0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.643+0000 7f9540bda740 1 -- 172.21.15.17:0/2848270372 shutdown_connections 2024-03-24T03:00:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.643+0000 7f9540bda740 1 --2- 172.21.15.17:0/2848270372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca72a0 0x559173ca9730 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.643+0000 7f9540bda740 1 --2- 172.21.15.17:0/2848270372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173ca6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.643+0000 7f9540bda740 1 --2- 172.21.15.17:0/2848270372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca1740 0x559173ca1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.643+0000 7f9540bda740 1 -- 172.21.15.17:0/2848270372 >> 172.21.15.17:0/2848270372 conn(0x559173c69a60 msgr2=0x559173c9d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9540bda740 1 -- 172.21.15.17:0/2848270372 shutdown_connections 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9540bda740 1 -- 172.21.15.17:0/2848270372 wait complete. 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9540bda740 1 -- start start 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9540bda740 1 Processor -- start 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9540bda740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca1740 0x559173cadbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9540bda740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca2130 0x559173cae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9540bda740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca72a0 0x559173cb5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9540bda740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559173caa010 con 0x559173ca1740 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9540bda740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559173bc2e90 con 0x559173ca72a0 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9540bda740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559173caa190 con 0x559173ca2130 2024-03-24T03:00:23.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9540bda740 10 client.0 _ll_register_callbacks cb 0x55917391aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f95377fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca1740 0x559173cadbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f95377fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca1740 0x559173cadbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35656/0 (socket says 172.21.15.17:35656) 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f95377fe640 1 -- 172.21.15.17:0/2315523959 learned_addr learned my addr 172.21.15.17:0/2315523959 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9536ffd640 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca2130 0x559173cae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.644+0000 7f9537fff640 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca72a0 0x559173cb5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f95377fe640 1 -- 172.21.15.17:0/2315523959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca2130 msgr2=0x559173cae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f95377fe640 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca2130 0x559173cae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f95377fe640 1 -- 172.21.15.17:0/2315523959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca72a0 msgr2=0x559173cb5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f95377fe640 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca72a0 0x559173cb5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f95377fe640 1 -- 172.21.15.17:0/2315523959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559173bcf9d0 con 0x559173ca1740 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f95377fe640 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca1740 0x559173cadbf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f95280126e0 tx=0x7f9528012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13912f56f33189df server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f9516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f9516ffd640 1 -- 172.21.15.17:0/2315523959 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9528012190 con 0x559173ca1740 2024-03-24T03:00:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f9516ffd640 1 -- 172.21.15.17:0/2315523959 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f9528004030 con 0x559173ca1740 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f9540bda740 1 -- 172.21.15.17:0/2315523959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95300bd9f0 con 0x559173ca1740 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f9516ffd640 1 -- 172.21.15.17:0/2315523959 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9528025050 con 0x559173ca1740 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f9540bda740 10 client.15255 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f9540bda740 1 -- 172.21.15.17:0/2315523959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55917391b2d0 con 0x559173ca1740 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.645+0000 7f9516ffd640 1 -- 172.21.15.17:0/2315523959 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95300bd9f0 con 0x559173ca1740 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.646+0000 7f9516ffd640 1 -- 172.21.15.17:0/2315523959 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f95280038c0 con 0x559173ca1740 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.646+0000 7f9540bda740 10 client.15255 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.646+0000 7f9540bda740 10 client.15255 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.646+0000 7f9540bda740 20 client.15255 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.646+0000 7f9540bda740 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x559173cba3d0 0x559173cb94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.646+0000 7f9540bda740 4 client.15255 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.646+0000 7f9540bda740 1 -- 172.21.15.17:0/2315523959 --> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559173c14160 con 0x559173cba3d0 2024-03-24T03:00:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.646+0000 7f9536ffd640 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x559173cba3d0 0x559173cb94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.646+0000 7f9516ffd640 10 client.15255 ms_handle_connect on v2:172.21.15.188:6834/3649219818 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.646+0000 7f9536ffd640 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x559173cba3d0 0x559173cb94a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b661ff16dde9089 server_cookie=976ea054e2083e72 in_seq=0 out_seq=0 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9516ffd640 1 -- 172.21.15.17:0/2315523959 <== mds.3 v2:172.21.15.188:6834/3649219818 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559173c14160 con 0x559173cba3d0 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9516ffd640 10 client.15255 handle_command_reply: tid=0 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9540bda740 1 client.15255 shutdown 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9540bda740 1 -- 172.21.15.17:0/2315523959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca1740 msgr2=0x559173cadbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9540bda740 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca1740 0x559173cadbf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f95280126e0 tx=0x7f9528012ba0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9540bda740 1 -- 172.21.15.17:0/2315523959 shutdown_connections 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9540bda740 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x559173cba3d0 0x559173cb94a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9540bda740 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca72a0 0x559173cb5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9540bda740 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca2130 0x559173cae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9540bda740 1 --2- 172.21.15.17:0/2315523959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca1740 0x559173cadbf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9516ffd640 0 client.15255 ms_handle_reset on v2:172.21.15.188:6834/3649219818 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9540bda740 1 -- 172.21.15.17:0/2315523959 >> 172.21.15.17:0/2315523959 conn(0x559173c69a60 msgr2=0x559173c9c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9540bda740 1 -- 172.21.15.17:0/2315523959 shutdown_connections 2024-03-24T03:00:23.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9540bda740 1 -- 172.21.15.17:0/2315523959 wait complete. 2024-03-24T03:00:23.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.647+0000 7f9540bda740 20 client.15255 trim_cache size 0 max 16384 2024-03-24T03:00:23.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f9540bda740 1 Processor -- start 2024-03-24T03:00:23.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f9540bda740 1 -- start start 2024-03-24T03:00:23.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f9540bda740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca1740 0x559173c9d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f9540bda740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173c9ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f9540bda740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca72a0 0x559173c9bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f9540bda740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559173cb9bf0 con 0x559173ca2130 2024-03-24T03:00:23.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f9540bda740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559173cba250 con 0x559173ca1740 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f9540bda740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559173cba8b0 con 0x559173ca72a0 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f9536ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173c9ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f95377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca1740 0x559173c9d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f9536ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173c9ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35672/0 (socket says 172.21.15.17:35672) 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f9536ffd640 1 -- 172.21.15.17:0/3391527940 learned_addr learned my addr 172.21.15.17:0/3391527940 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.664+0000 7f9537fff640 1 --2- 172.21.15.17:0/3391527940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca72a0 0x559173c9bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.665+0000 7f95377fe640 1 -- 172.21.15.17:0/3391527940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca72a0 msgr2=0x559173c9bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.665+0000 7f95377fe640 1 --2- 172.21.15.17:0/3391527940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca72a0 0x559173c9bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.665+0000 7f95377fe640 1 -- 172.21.15.17:0/3391527940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 msgr2=0x559173c9ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.665+0000 7f95377fe640 1 --2- 172.21.15.17:0/3391527940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173c9ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.665+0000 7f95377fe640 1 -- 172.21.15.17:0/3391527940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591739e56b0 con 0x559173ca1740 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.665+0000 7f9536ffd640 1 --2- 172.21.15.17:0/3391527940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173c9ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.665+0000 7f95377fe640 1 --2- 172.21.15.17:0/3391527940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca1740 0x559173c9d2e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9528001eb0 tx=0x7f9528005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e134df4127a0d46 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.665+0000 7f951d7fa640 1 -- 172.21.15.17:0/3391527940 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9528022040 con 0x559173ca1740 2024-03-24T03:00:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.665+0000 7f951d7fa640 1 -- 172.21.15.17:0/3391527940 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95280050f0 con 0x559173ca1740 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.665+0000 7f951d7fa640 1 -- 172.21.15.17:0/3391527940 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952800fa80 con 0x559173ca1740 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 -- 172.21.15.17:0/3391527940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca1740 msgr2=0x559173c9d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 --2- 172.21.15.17:0/3391527940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca1740 0x559173c9d2e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9528001eb0 tx=0x7f9528005d30 comp rx=0 tx=0).stop 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 -- 172.21.15.17:0/3391527940 shutdown_connections 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 --2- 172.21.15.17:0/3391527940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca72a0 0x559173c9bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 --2- 172.21.15.17:0/3391527940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173c9ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 --2- 172.21.15.17:0/3391527940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca1740 0x559173c9d2e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 -- 172.21.15.17:0/3391527940 >> 172.21.15.17:0/3391527940 conn(0x559173c69a60 msgr2=0x559173c9cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 -- 172.21.15.17:0/3391527940 shutdown_connections 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 -- 172.21.15.17:0/3391527940 wait complete. 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 -- start start 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 Processor -- start 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca1740 0x559173cac150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173cac690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca72a0 0x559173cad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559173cbb570 con 0x559173ca2130 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559173cbaf10 con 0x559173ca72a0 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559173cbbbd0 con 0x559173ca1740 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9540bda740 10 client.0 _ll_register_callbacks cb 0x7f9528009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9536ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173cac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f95377fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca1740 0x559173cac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9536ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173cac690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35684/0 (socket says 172.21.15.17:35684) 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9536ffd640 1 -- 172.21.15.17:0/4270688743 learned_addr learned my addr 172.21.15.17:0/4270688743 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.666+0000 7f9537fff640 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca72a0 0x559173cad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9536ffd640 1 -- 172.21.15.17:0/4270688743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca1740 msgr2=0x559173cac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9536ffd640 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca1740 0x559173cac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9536ffd640 1 -- 172.21.15.17:0/4270688743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca72a0 msgr2=0x559173cad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9536ffd640 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca72a0 0x559173cad6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9536ffd640 1 -- 172.21.15.17:0/4270688743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559173cadc90 con 0x559173ca2130 2024-03-24T03:00:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9537fff640 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca72a0 0x559173cad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f95377fe640 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca1740 0x559173cac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9536ffd640 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173cac690 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f952c014fe0 tx=0x7f952c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b9888eec855f92 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9516ffd640 1 -- 172.21.15.17:0/4270688743 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c02c2e0 con 0x559173ca2130 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9516ffd640 1 -- 172.21.15.17:0/4270688743 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f952c005020 con 0x559173ca2130 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9540bda740 1 -- 172.21.15.17:0/4270688743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95280050f0 con 0x559173ca2130 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9516ffd640 1 -- 172.21.15.17:0/4270688743 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f952c005220 con 0x559173ca2130 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9540bda740 10 client.15261 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9540bda740 1 -- 172.21.15.17:0/4270688743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559173d17dc0 con 0x559173ca2130 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.667+0000 7f9516ffd640 1 -- 172.21.15.17:0/4270688743 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95280050f0 con 0x559173ca2130 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9516ffd640 1 -- 172.21.15.17:0/4270688743 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f952c01f070 con 0x559173ca2130 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 10 client.15261 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 10 client.15261 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 20 client.15261 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x559173d1c430 0x559173d17ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 4 client.15261 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 1 -- 172.21.15.17:0/4270688743 --> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559173cb9a40 con 0x559173d1c430 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f95377fe640 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x559173d1c430 0x559173d17ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f95377fe640 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x559173d1c430 0x559173d17ff0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cdd9144f1627af79 server_cookie=323125ed289df44a in_seq=0 out_seq=0 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9516ffd640 10 client.15261 ms_handle_connect on v2:172.21.15.188:6834/3649219818 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9516ffd640 1 -- 172.21.15.17:0/4270688743 <== mds.3 v2:172.21.15.188:6834/3649219818 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x559173cb9a40 con 0x559173d1c430 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9516ffd640 10 client.15261 handle_command_reply: tid=0 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 1 client.15261 shutdown 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 1 -- 172.21.15.17:0/4270688743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 msgr2=0x559173cac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173cac690 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f952c014fe0 tx=0x7f952c004730 comp rx=0 tx=0).stop 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 1 -- 172.21.15.17:0/4270688743 shutdown_connections 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x559173d1c430 0x559173d17ff0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559173ca72a0 0x559173cad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.668+0000 7f9540bda740 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559173ca2130 0x559173cac690 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f9516ffd640 0 client.15261 ms_handle_reset on v2:172.21.15.188:6834/3649219818 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f9540bda740 1 --2- 172.21.15.17:0/4270688743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559173ca1740 0x559173cac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f9540bda740 1 -- 172.21.15.17:0/4270688743 >> 172.21.15.17:0/4270688743 conn(0x559173c69a60 msgr2=0x559173ca02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f9540bda740 1 -- 172.21.15.17:0/4270688743 shutdown_connections 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f9540bda740 1 -- 172.21.15.17:0/4270688743 wait complete. 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f9540bda740 20 client.15261 trim_cache size 0 max 16384 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 3, 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24493, 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-24T03:00:23.675 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 86.611742355999993, 2024-03-24T03:00:23.676 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 28, 2024-03-24T03:00:23.676 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 91, 2024-03-24T03:00:23.676 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-03-24T03:00:23.676 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 129.91534486399999 2024-03-24T03:00:23.676 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:00:23.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f953dff3640 1 -- 172.21.15.17:0/2989358456 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f95200a2800 msgr2=0x7f95200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f953dff3640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f95200a2800 0x7f95200a4cc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f952c013570 tx=0x7f952c009f90 comp rx=0 tx=0).stop 2024-03-24T03:00:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f953dff3640 1 -- 172.21.15.17:0/2989358456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9538102150 msgr2=0x7f953819de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f953dff3640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9538102150 0x7f953819de30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9530036040 tx=0x7f95300337b0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f953dff3640 1 -- 172.21.15.17:0/2989358456 shutdown_connections 2024-03-24T03:00:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f953dff3640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f95200a2800 0x7f95200a4cc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f953dff3640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9538102150 0x7f953819de30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f953dff3640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9538101790 0x7f9538196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f953dff3640 1 --2- 172.21.15.17:0/2989358456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95381005a0 0x7f9538196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f953dff3640 1 -- 172.21.15.17:0/2989358456 >> 172.21.15.17:0/2989358456 conn(0x7f95380fbb30 msgr2=0x7f9538104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f953dff3640 1 -- 172.21.15.17:0/2989358456 shutdown_connections 2024-03-24T03:00:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.669+0000 7f953dff3640 1 -- 172.21.15.17:0/2989358456 wait complete. 2024-03-24T03:00:23.679 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-24T03:00:23.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 -- 172.21.15.17:0/1929558171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c100110 msgr2=0x7fd82c109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 --2- 172.21.15.17:0/1929558171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c100110 0x7fd82c109cc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd824009ca0 tx=0x7fd82402c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 -- 172.21.15.17:0/1929558171 shutdown_connections 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 --2- 172.21.15.17:0/1929558171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c100110 0x7fd82c109cc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 --2- 172.21.15.17:0/1929558171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82c0ff750 0x7fd82c0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 --2- 172.21.15.17:0/1929558171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82c102ce0 0x7fd82c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 -- 172.21.15.17:0/1929558171 >> 172.21.15.17:0/1929558171 conn(0x7fd82c0fb2d0 msgr2=0x7fd82c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 -- 172.21.15.17:0/1929558171 shutdown_connections 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 -- 172.21.15.17:0/1929558171 wait complete. 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 Processor -- start 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 -- start start 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c0ff750 0x7fd82c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.784+0000 7fd83300b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82c100110 0x7fd82c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd83300b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82c102ce0 0x7fd82c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd83300b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c10d310 con 0x7fd82c0ff750 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd83300b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c10d190 con 0x7fd82c100110 2024-03-24T03:00:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd83300b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c10d490 con 0x7fd82c102ce0 2024-03-24T03:00:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd830da7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c0ff750 0x7fd82c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd830da7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c0ff750 0x7fd82c1978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35702/0 (socket says 172.21.15.17:35702) 2024-03-24T03:00:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd830da7640 1 -- 172.21.15.17:0/1125563960 learned_addr learned my addr 172.21.15.17:0/1125563960 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd82bfff640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82c100110 0x7fd82c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd8315a8640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82c102ce0 0x7fd82c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd830da7640 1 -- 172.21.15.17:0/1125563960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82c102ce0 msgr2=0x7fd82c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd830da7640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82c102ce0 0x7fd82c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd830da7640 1 -- 172.21.15.17:0/1125563960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82c100110 msgr2=0x7fd82c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd830da7640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82c100110 0x7fd82c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd830da7640 1 -- 172.21.15.17:0/1125563960 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82c19f510 con 0x7fd82c0ff750 2024-03-24T03:00:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd830da7640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c0ff750 0x7fd82c1978f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd82000e9c0 tx=0x7fd82000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd829ffb640 1 -- 172.21.15.17:0/1125563960 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000cd80 con 0x7fd82c0ff750 2024-03-24T03:00:23.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd829ffb640 1 -- 172.21.15.17:0/1125563960 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8200035f0 con 0x7fd82c0ff750 2024-03-24T03:00:23.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd83300b640 1 -- 172.21.15.17:0/1125563960 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82c19f800 con 0x7fd82c0ff750 2024-03-24T03:00:23.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd829ffb640 1 -- 172.21.15.17:0/1125563960 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820010690 con 0x7fd82c0ff750 2024-03-24T03:00:23.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.785+0000 7fd83300b640 1 -- 172.21.15.17:0/1125563960 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82c19fd50 con 0x7fd82c0ff750 2024-03-24T03:00:23.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd829ffb640 1 -- 172.21.15.17:0/1125563960 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd8200037a0 con 0x7fd82c0ff750 2024-03-24T03:00:23.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd835bf5740 1 Processor -- start 2024-03-24T03:00:23.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd835bf5740 1 -- start start 2024-03-24T03:00:23.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd835bf5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172ad9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd835bf5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 0x558172adec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd835bf5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172adf2a0 0x558172ae1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd835bf5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172a3ec20 con 0x558172ada130 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd835bf5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581729e7a40 con 0x558172ad9740 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd835bf5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172a3acc0 con 0x558172adf2a0 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd829ffb640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd81c0a2800 0x7fd81c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd830da7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 0x558172adec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd8315a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172ad9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd830da7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 0x558172adec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35718/0 (socket says 172.21.15.17:35718) 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd82bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172adf2a0 0x558172ae1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd830da7640 1 -- 172.21.15.17:0/1204381174 learned_addr learned my addr 172.21.15.17:0/1204381174 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd8315a8640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd81c0a2800 0x7fd81c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.788+0000 7fd829ffb640 1 -- 172.21.15.17:0/1125563960 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fd8200cf140 con 0x7fd82c0ff750 2024-03-24T03:00:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.789+0000 7fd830da7640 1 -- 172.21.15.17:0/1204381174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172adf2a0 msgr2=0x558172ae1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.789+0000 7fd830da7640 1 --2- 172.21.15.17:0/1204381174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172adf2a0 0x558172ae1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.789+0000 7fd830da7640 1 -- 172.21.15.17:0/1204381174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 msgr2=0x558172ad9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.789+0000 7fd830da7640 1 --2- 172.21.15.17:0/1204381174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172ad9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.789+0000 7fd830da7640 1 -- 172.21.15.17:0/1204381174 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558172a09f90 con 0x558172ada130 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.789+0000 7fd82bfff640 1 --2- 172.21.15.17:0/1204381174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172adf2a0 0x558172ae1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.789+0000 7fd830da7640 1 --2- 172.21.15.17:0/1204381174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 0x558172adec30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd820000c00 tx=0x7fd82006e4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e66b2f09765b6d server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.789+0000 7fd8315a8640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd81c0a2800 0x7fd81c0a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd824005e30 tx=0x7fd824032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.789+0000 7fd816ffd640 1 -- 172.21.15.17:0/1204381174 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82009b110 con 0x558172ada130 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.789+0000 7fd816ffd640 1 -- 172.21.15.17:0/1204381174 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd82006d4a0 con 0x558172ada130 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.789+0000 7fd816ffd640 1 -- 172.21.15.17:0/1204381174 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820003b30 con 0x558172ada130 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 -- 172.21.15.17:0/1204381174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 msgr2=0x558172adec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 --2- 172.21.15.17:0/1204381174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 0x558172adec30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd820000c00 tx=0x7fd82006e4e0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 -- 172.21.15.17:0/1204381174 shutdown_connections 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 --2- 172.21.15.17:0/1204381174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172adf2a0 0x558172ae1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 --2- 172.21.15.17:0/1204381174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 0x558172adec30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 --2- 172.21.15.17:0/1204381174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172ad9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 -- 172.21.15.17:0/1204381174 >> 172.21.15.17:0/1204381174 conn(0x558172aa1a60 msgr2=0x558172ad50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 -- 172.21.15.17:0/1204381174 shutdown_connections 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 -- 172.21.15.17:0/1204381174 wait complete. 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 -- start start 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 Processor -- start 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ad9740 0x558172ae3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ada130 0x558172ae4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172adf2a0 0x558172aeb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172ae2250 con 0x558172ad9740 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581729fae90 con 0x558172ada130 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172ae23d0 con 0x558172adf2a0 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.790+0000 7fd835bf5740 10 client.0 _ll_register_callbacks cb 0x558172752ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd830da7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ad9740 0x558172ae3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd830da7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ad9740 0x558172ae3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35720/0 (socket says 172.21.15.17:35720) 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd830da7640 1 -- 172.21.15.17:0/3777609300 learned_addr learned my addr 172.21.15.17:0/3777609300 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd8315a8640 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172adf2a0 0x558172aeb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd82bfff640 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ada130 0x558172ae4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd830da7640 1 -- 172.21.15.17:0/3777609300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172adf2a0 msgr2=0x558172aeb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd830da7640 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172adf2a0 0x558172aeb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd830da7640 1 -- 172.21.15.17:0/3777609300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ada130 msgr2=0x558172ae4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd830da7640 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ada130 0x558172ae4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd830da7640 1 -- 172.21.15.17:0/3777609300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558172a079d0 con 0x558172ad9740 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd830da7640 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ad9740 0x558172ae3c30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd82009b740 tx=0x7fd820086290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1a772214a09b2c server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd80bfff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd80bfff640 1 -- 172.21.15.17:0/3777609300 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82009b7b0 con 0x558172ad9740 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd80bfff640 1 -- 172.21.15.17:0/3777609300 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fd8200a3ac0 con 0x558172ad9740 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd835bf5740 1 -- 172.21.15.17:0/3777609300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558172a4c160 con 0x558172ad9740 2024-03-24T03:00:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd80bfff640 1 -- 172.21.15.17:0/3777609300 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd82009f900 con 0x558172ad9740 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd835bf5740 10 client.15273 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd835bf5740 1 -- 172.21.15.17:0/3777609300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5581727532d0 con 0x558172ad9740 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.791+0000 7fd80bfff640 1 -- 172.21.15.17:0/3777609300 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558172a4c160 con 0x558172ad9740 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.792+0000 7fd80bfff640 1 -- 172.21.15.17:0/3777609300 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd8200c8020 con 0x558172ad9740 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.792+0000 7fd835bf5740 10 client.15273 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.792+0000 7fd835bf5740 10 client.15273 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.792+0000 7fd835bf5740 20 client.15273 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.792+0000 7fd835bf5740 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x558172aefe70 0x558172af42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.792+0000 7fd835bf5740 4 client.15273 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.792+0000 7fd835bf5740 1 -- 172.21.15.17:0/3777609300 --> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558172a464b0 con 0x558172aefe70 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.792+0000 7fd82bfff640 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x558172aefe70 0x558172af42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.792+0000 7fd80bfff640 10 client.15273 ms_handle_connect on v2:172.21.15.188:6836/3512350624 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.792+0000 7fd82bfff640 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x558172aefe70 0x558172af42f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c44233bf1ba10c03 server_cookie=cc82f2f6a031c105 in_seq=0 out_seq=0 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd80bfff640 1 -- 172.21.15.17:0/3777609300 <== mds.2 v2:172.21.15.188:6836/3512350624 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558172a464b0 con 0x558172aefe70 2024-03-24T03:00:23.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd80bfff640 10 client.15273 handle_command_reply: tid=0 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd835bf5740 1 client.15273 shutdown 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd835bf5740 1 -- 172.21.15.17:0/3777609300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ad9740 msgr2=0x558172ae3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd835bf5740 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ad9740 0x558172ae3c30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd82009b740 tx=0x7fd820086290 comp rx=0 tx=0).stop 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd835bf5740 1 -- 172.21.15.17:0/3777609300 shutdown_connections 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd835bf5740 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172adf2a0 0x558172aeb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd835bf5740 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x558172aefe70 0x558172af42f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd835bf5740 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ada130 0x558172ae4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd835bf5740 1 --2- 172.21.15.17:0/3777609300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ad9740 0x558172ae3c30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd80bfff640 0 client.15273 ms_handle_reset on v2:172.21.15.188:6836/3512350624 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd835bf5740 1 -- 172.21.15.17:0/3777609300 >> 172.21.15.17:0/3777609300 conn(0x558172aa1a60 msgr2=0x558172ad42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd835bf5740 1 -- 172.21.15.17:0/3777609300 shutdown_connections 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd835bf5740 1 -- 172.21.15.17:0/3777609300 wait complete. 2024-03-24T03:00:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.793+0000 7fd835bf5740 20 client.15273 trim_cache size 0 max 16384 2024-03-24T03:00:23.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.812+0000 7fd835bf5740 1 Processor -- start 2024-03-24T03:00:23.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.812+0000 7fd835bf5740 1 -- start start 2024-03-24T03:00:23.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.812+0000 7fd835bf5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172aa1a60 0x558172af9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.812+0000 7fd835bf5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172aeb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.812+0000 7fd835bf5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172ada130 0x558172ad8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.812+0000 7fd835bf5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172aef690 con 0x558172aa1a60 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.812+0000 7fd835bf5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172aefcf0 con 0x558172ad9740 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.812+0000 7fd835bf5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172af0350 con 0x558172ada130 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.812+0000 7fd830da7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172aa1a60 0x558172af9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.812+0000 7fd830da7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172aa1a60 0x558172af9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35728/0 (socket says 172.21.15.17:35728) 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.812+0000 7fd830da7640 1 -- 172.21.15.17:0/3548608146 learned_addr learned my addr 172.21.15.17:0/3548608146 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.813+0000 7fd82bfff640 1 --2- 172.21.15.17:0/3548608146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172aeb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.813+0000 7fd8315a8640 1 --2- 172.21.15.17:0/3548608146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172ada130 0x558172ad8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.813+0000 7fd830da7640 1 -- 172.21.15.17:0/3548608146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172ada130 msgr2=0x558172ad8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.813+0000 7fd830da7640 1 --2- 172.21.15.17:0/3548608146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172ada130 0x558172ad8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.813+0000 7fd830da7640 1 -- 172.21.15.17:0/3548608146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 msgr2=0x558172aeb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.813+0000 7fd830da7640 1 --2- 172.21.15.17:0/3548608146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172aeb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.813+0000 7fd830da7640 1 -- 172.21.15.17:0/3548608146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55817281d6b0 con 0x558172aa1a60 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.813+0000 7fd82bfff640 1 --2- 172.21.15.17:0/3548608146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172aeb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.813+0000 7fd830da7640 1 --2- 172.21.15.17:0/3548608146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172aa1a60 0x558172af9320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd82009b4b0 tx=0x7fd8200a4010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fc366934c21d6a server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.813+0000 7fd8167fc640 1 -- 172.21.15.17:0/3548608146 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82009b7b0 con 0x558172aa1a60 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.813+0000 7fd8167fc640 1 -- 172.21.15.17:0/3548608146 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd82009acf0 con 0x558172aa1a60 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.813+0000 7fd8167fc640 1 -- 172.21.15.17:0/3548608146 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8200a3ac0 con 0x558172aa1a60 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.814+0000 7fd835bf5740 1 -- 172.21.15.17:0/3548608146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172aa1a60 msgr2=0x558172af9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.814+0000 7fd835bf5740 1 --2- 172.21.15.17:0/3548608146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172aa1a60 0x558172af9320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd82009b4b0 tx=0x7fd8200a4010 comp rx=0 tx=0).stop 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.814+0000 7fd835bf5740 1 -- 172.21.15.17:0/3548608146 shutdown_connections 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.814+0000 7fd835bf5740 1 --2- 172.21.15.17:0/3548608146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172ada130 0x558172ad8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.814+0000 7fd835bf5740 1 --2- 172.21.15.17:0/3548608146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172aeb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.814+0000 7fd835bf5740 1 --2- 172.21.15.17:0/3548608146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172aa1a60 0x558172af9320 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.814+0000 7fd835bf5740 1 -- 172.21.15.17:0/3548608146 >> 172.21.15.17:0/3548608146 conn(0x558172adaf10 msgr2=0x558172ae3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.814+0000 7fd835bf5740 1 -- 172.21.15.17:0/3548608146 shutdown_connections 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.814+0000 7fd835bf5740 1 -- 172.21.15.17:0/3548608146 wait complete. 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.814+0000 7fd835bf5740 1 -- start start 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.814+0000 7fd835bf5740 1 Processor -- start 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd835bf5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172aa1a60 0x558172ae82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd835bf5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172ae8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd835bf5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 0x558172ae9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd835bf5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172af1010 con 0x558172ada130 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd835bf5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172af09b0 con 0x558172ad9740 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd835bf5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558172af1670 con 0x558172aa1a60 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd835bf5740 10 client.0 _ll_register_callbacks cb 0x7fd820005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd8315a8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 0x558172ae9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd8315a8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 0x558172ae9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35744/0 (socket says 172.21.15.17:35744) 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd830da7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172aa1a60 0x558172ae82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd8315a8640 1 -- 172.21.15.17:0/361141643 learned_addr learned my addr 172.21.15.17:0/361141643 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd82bfff640 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172ae8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd8315a8640 1 -- 172.21.15.17:0/361141643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172aa1a60 msgr2=0x558172ae82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd8315a8640 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172aa1a60 0x558172ae82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd8315a8640 1 -- 172.21.15.17:0/361141643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 msgr2=0x558172ae8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd8315a8640 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172ae8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd8315a8640 1 -- 172.21.15.17:0/361141643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558172ae9e20 con 0x558172ada130 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd82bfff640 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172ae8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd8315a8640 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 0x558172ae9830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd824009ca0 tx=0x7fd8240095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9597c5fe6832dc0e server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd80bfff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd80bfff640 1 -- 172.21.15.17:0/361141643 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824039040 con 0x558172ada130 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd80bfff640 1 -- 172.21.15.17:0/361141643 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fd824004450 con 0x558172ada130 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd835bf5740 1 -- 172.21.15.17:0/361141643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd82009acf0 con 0x558172ada130 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.815+0000 7fd80bfff640 1 -- 172.21.15.17:0/361141643 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd824004650 con 0x558172ada130 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.816+0000 7fd835bf5740 10 client.15279 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:23.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.816+0000 7fd835bf5740 1 -- 172.21.15.17:0/361141643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558172aeb190 con 0x558172ada130 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.816+0000 7fd80bfff640 1 -- 172.21.15.17:0/361141643 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd82009acf0 con 0x558172ada130 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.816+0000 7fd80bfff640 1 -- 172.21.15.17:0/361141643 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd824005620 con 0x558172ada130 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.816+0000 7fd835bf5740 10 client.15279 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.816+0000 7fd835bf5740 10 client.15279 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.816+0000 7fd835bf5740 20 client.15279 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.816+0000 7fd835bf5740 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x558172adf2a0 0x558172b4d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.816+0000 7fd835bf5740 4 client.15279 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.816+0000 7fd835bf5740 1 -- 172.21.15.17:0/361141643 --> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5581729c49a0 con 0x558172adf2a0 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.816+0000 7fd830da7640 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x558172adf2a0 0x558172b4d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd830da7640 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x558172adf2a0 0x558172b4d420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f0098efa47121f5 server_cookie=b2911ec9e0b708bf in_seq=0 out_seq=0 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd80bfff640 10 client.15279 ms_handle_connect on v2:172.21.15.188:6836/3512350624 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd80bfff640 1 -- 172.21.15.17:0/361141643 <== mds.2 v2:172.21.15.188:6836/3512350624 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x5581729c49a0 con 0x558172adf2a0 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd80bfff640 10 client.15279 handle_command_reply: tid=0 2024-03-24T03:00:23.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd835bf5740 1 client.15279 shutdown 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd835bf5740 1 -- 172.21.15.17:0/361141643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 msgr2=0x558172ae9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd835bf5740 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 0x558172ae9830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd824009ca0 tx=0x7fd8240095c0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd835bf5740 1 -- 172.21.15.17:0/361141643 shutdown_connections 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd835bf5740 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558172ada130 0x558172ae9830 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd835bf5740 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x558172adf2a0 0x558172b4d420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd835bf5740 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558172ad9740 0x558172ae8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd835bf5740 1 --2- 172.21.15.17:0/361141643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558172aa1a60 0x558172ae82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd80bfff640 0 client.15279 ms_handle_reset on v2:172.21.15.188:6836/3512350624 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd835bf5740 1 -- 172.21.15.17:0/361141643 >> 172.21.15.17:0/361141643 conn(0x558172adaf10 msgr2=0x558172ad8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd835bf5740 1 -- 172.21.15.17:0/361141643 shutdown_connections 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd835bf5740 1 -- 172.21.15.17:0/361141643 wait complete. 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.817+0000 7fd835bf5740 20 client.15279 trim_cache size 0 max 16384 2024-03-24T03:00:23.823 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:00:23.824 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T03:00:23.824 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 2, 2024-03-24T03:00:23.824 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24511, 2024-03-24T03:00:23.824 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-24T03:00:23.824 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-24T03:00:23.824 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-24T03:00:23.824 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 87.768216568, 2024-03-24T03:00:23.824 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 28, 2024-03-24T03:00:23.824 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 91, 2024-03-24T03:00:23.824 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-03-24T03:00:23.824 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 127.181315275 2024-03-24T03:00:23.824 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:00:23.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.818+0000 7fd83300b640 1 -- 172.21.15.17:0/1125563960 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd81c0a2800 msgr2=0x7fd81c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.818+0000 7fd83300b640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd81c0a2800 0x7fd81c0a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd824005e30 tx=0x7fd824032590 comp rx=0 tx=0).stop 2024-03-24T03:00:23.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.818+0000 7fd83300b640 1 -- 172.21.15.17:0/1125563960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c0ff750 msgr2=0x7fd82c1978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.818+0000 7fd83300b640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c0ff750 0x7fd82c1978f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd82000e9c0 tx=0x7fd82000ee80 comp rx=0 tx=0).stop 2024-03-24T03:00:23.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.818+0000 7fd83300b640 1 -- 172.21.15.17:0/1125563960 shutdown_connections 2024-03-24T03:00:23.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.818+0000 7fd83300b640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd81c0a2800 0x7fd81c0a4cc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.818+0000 7fd83300b640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82c102ce0 0x7fd82c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.818+0000 7fd83300b640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82c100110 0x7fd82c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.818+0000 7fd83300b640 1 --2- 172.21.15.17:0/1125563960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c0ff750 0x7fd82c1978f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.818+0000 7fd83300b640 1 -- 172.21.15.17:0/1125563960 >> 172.21.15.17:0/1125563960 conn(0x7fd82c0fb2d0 msgr2=0x7fd82c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.818+0000 7fd83300b640 1 -- 172.21.15.17:0/1125563960 shutdown_connections 2024-03-24T03:00:23.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.818+0000 7fd83300b640 1 -- 172.21.15.17:0/1125563960 wait complete. 2024-03-24T03:00:23.828 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-24T03:00:23.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.932+0000 7fc8b9977640 1 -- 172.21.15.17:0/567766233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b4107420 msgr2=0x7fc8b4109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.932+0000 7fc8b9977640 1 --2- 172.21.15.17:0/567766233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b4107420 0x7fc8b4109810 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009ca0 tx=0x7fc8a402c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.932+0000 7fc8b9977640 1 -- 172.21.15.17:0/567766233 shutdown_connections 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.932+0000 7fc8b9977640 1 --2- 172.21.15.17:0/567766233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b4107420 0x7fc8b4109810 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.932+0000 7fc8b9977640 1 --2- 172.21.15.17:0/567766233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b4104af0 0x7fc8b4106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.932+0000 7fc8b9977640 1 --2- 172.21.15.17:0/567766233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8b41021c0 0x7fc8b41045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.932+0000 7fc8b9977640 1 -- 172.21.15.17:0/567766233 >> 172.21.15.17:0/567766233 conn(0x7fc8b40fbb30 msgr2=0x7fc8b40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.933+0000 7fc8b9977640 1 -- 172.21.15.17:0/567766233 shutdown_connections 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.933+0000 7fc8b9977640 1 -- 172.21.15.17:0/567766233 wait complete. 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.933+0000 7fc8b9977640 1 Processor -- start 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.933+0000 7fc8b9977640 1 -- start start 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.933+0000 7fc8b9977640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b41021c0 0x7fc8b41979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.933+0000 7fc8b9977640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b4104af0 0x7fc8b4197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.933+0000 7fc8b9977640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8b4107420 0x7fc8b419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.933+0000 7fc8b9977640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b410d320 con 0x7fc8b41021c0 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.933+0000 7fc8b9977640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b410d1a0 con 0x7fc8b4104af0 2024-03-24T03:00:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.933+0000 7fc8b9977640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b410d4a0 con 0x7fc8b4107420 2024-03-24T03:00:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.933+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b41021c0 0x7fc8b41979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b41021c0 0x7fc8b41979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35758/0 (socket says 172.21.15.17:35758) 2024-03-24T03:00:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b2ffd640 1 -- 172.21.15.17:0/1849782475 learned_addr learned my addr 172.21.15.17:0/1849782475 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b27fc640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b4104af0 0x7fc8b4197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8b4107420 0x7fc8b419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b2ffd640 1 -- 172.21.15.17:0/1849782475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8b4107420 msgr2=0x7fc8b419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b2ffd640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8b4107420 0x7fc8b419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b2ffd640 1 -- 172.21.15.17:0/1849782475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b4104af0 msgr2=0x7fc8b4197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b2ffd640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b4104af0 0x7fc8b4197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b2ffd640 1 -- 172.21.15.17:0/1849782475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b419f5e0 con 0x7fc8b41021c0 2024-03-24T03:00:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b2ffd640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b41021c0 0x7fc8b41979c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00e9f0 tx=0x7fc8ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b8975640 1 -- 172.21.15.17:0/1849782475 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac00cd80 con 0x7fc8b41021c0 2024-03-24T03:00:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b9977640 1 -- 172.21.15.17:0/1849782475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b419f8d0 con 0x7fc8b41021c0 2024-03-24T03:00:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b8975640 1 -- 172.21.15.17:0/1849782475 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8ac010910 con 0x7fc8b41021c0 2024-03-24T03:00:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b8975640 1 -- 172.21.15.17:0/1849782475 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac0188f0 con 0x7fc8b41021c0 2024-03-24T03:00:23.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.934+0000 7fc8b9977640 1 -- 172.21.15.17:0/1849782475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b419fe70 con 0x7fc8b41021c0 2024-03-24T03:00:23.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.936+0000 7fc8b8975640 1 -- 172.21.15.17:0/1849782475 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc8ac010ac0 con 0x7fc8b41021c0 2024-03-24T03:00:23.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8bc565740 1 Processor -- start 2024-03-24T03:00:23.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8bc565740 1 -- start start 2024-03-24T03:00:23.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8bc565740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fee740 0x557f63feeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8bc565740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fef130 0x557f63ff3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8bc565740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f63ff6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8bc565740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f63f53c20 con 0x557f63ff42a0 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8bc565740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f63efca40 con 0x557f63fee740 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8bc565740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f63f4fcc0 con 0x557f63fef130 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8b8975640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8a00a2800 0x7fc8a00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8b27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f63ff6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fef130 0x557f63ff3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fee740 0x557f63feeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fee740 0x557f63feeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:44726/0 (socket says 172.21.15.17:44726) 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8b37fe640 1 -- 172.21.15.17:0/2778307116 learned_addr learned my addr 172.21.15.17:0/2778307116 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8b27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f63ff6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35766/0 (socket says 172.21.15.17:35766) 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.937+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8a00a2800 0x7fc8a00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.938+0000 7fc8b8975640 1 -- 172.21.15.17:0/1849782475 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fc8ac0cf160 con 0x7fc8b41021c0 2024-03-24T03:00:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.938+0000 7fc8b37fe640 1 -- 172.21.15.17:0/2778307116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fef130 msgr2=0x557f63ff3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.938+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/2778307116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fef130 0x557f63ff3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.938+0000 7fc8b37fe640 1 -- 172.21.15.17:0/2778307116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 msgr2=0x557f63ff6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.938+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/2778307116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f63ff6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.938+0000 7fc8b37fe640 1 -- 172.21.15.17:0/2778307116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f63f1ef90 con 0x557f63fee740 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.938+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/2778307116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fee740 0x557f63feeb60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4032420 tx=0x7fc8a4032f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f92e5d2cc13d738 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.938+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8a00a2800 0x7fc8a00a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009430 tx=0x7fc8a4042000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.938+0000 7fc8a9ffb640 1 -- 172.21.15.17:0/2778307116 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4051070 con 0x557f63fee740 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.938+0000 7fc8a9ffb640 1 -- 172.21.15.17:0/2778307116 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8a4004b40 con 0x557f63fee740 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.938+0000 7fc8a9ffb640 1 -- 172.21.15.17:0/2778307116 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4039040 con 0x557f63fee740 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.939+0000 7fc8bc565740 1 -- 172.21.15.17:0/2778307116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fee740 msgr2=0x557f63feeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.939+0000 7fc8bc565740 1 --2- 172.21.15.17:0/2778307116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fee740 0x557f63feeb60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4032420 tx=0x7fc8a4032f60 comp rx=0 tx=0).stop 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.939+0000 7fc8bc565740 1 -- 172.21.15.17:0/2778307116 shutdown_connections 2024-03-24T03:00:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.939+0000 7fc8bc565740 1 --2- 172.21.15.17:0/2778307116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f63ff6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.939+0000 7fc8bc565740 1 --2- 172.21.15.17:0/2778307116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fef130 0x557f63ff3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.939+0000 7fc8bc565740 1 --2- 172.21.15.17:0/2778307116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fee740 0x557f63feeb60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.939+0000 7fc8bc565740 1 -- 172.21.15.17:0/2778307116 >> 172.21.15.17:0/2778307116 conn(0x557f63fb6a60 msgr2=0x557f63fea0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.939+0000 7fc8bc565740 1 -- 172.21.15.17:0/2778307116 shutdown_connections 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.939+0000 7fc8bc565740 1 -- 172.21.15.17:0/2778307116 wait complete. 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.939+0000 7fc8bc565740 1 -- start start 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.939+0000 7fc8bc565740 1 Processor -- start 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8bc565740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fee740 0x557f63ff8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8bc565740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fef130 0x557f63ff91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8bc565740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f64000170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8bc565740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f63ff71f0 con 0x557f63ff42a0 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8bc565740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f63f0fe90 con 0x557f63fee740 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8bc565740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f63ff7370 con 0x557f63fef130 2024-03-24T03:00:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8bc565740 10 client.0 _ll_register_callbacks cb 0x557f63c67ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f64000170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f64000170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35774/0 (socket says 172.21.15.17:35774) 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8b27fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fef130 0x557f63ff91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fee740 0x557f63ff8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8b37fe640 1 -- 172.21.15.17:0/2117161896 learned_addr learned my addr 172.21.15.17:0/2117161896 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8b37fe640 1 -- 172.21.15.17:0/2117161896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fef130 msgr2=0x557f63ff91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/2117161896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fef130 0x557f63ff91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8b37fe640 1 -- 172.21.15.17:0/2117161896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fee740 msgr2=0x557f63ff8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/2117161896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fee740 0x557f63ff8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8b37fe640 1 -- 172.21.15.17:0/2117161896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f63f1c9d0 con 0x557f63ff42a0 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8b27fc640 1 --2- 172.21.15.17:0/2117161896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fef130 0x557f63ff91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/2117161896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f64000170 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4037040 tx=0x7fc8a4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e73e679dc72d738 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc892ffd640 1 -- 172.21.15.17:0/2117161896 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a405a060 con 0x557f63ff42a0 2024-03-24T03:00:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc892ffd640 1 -- 172.21.15.17:0/2117161896 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fc8a4055460 con 0x557f63ff42a0 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.940+0000 7fc8bc565740 1 -- 172.21.15.17:0/2117161896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8a4004b40 con 0x557f63ff42a0 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.941+0000 7fc892ffd640 1 -- 172.21.15.17:0/2117161896 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8a4061050 con 0x557f63ff42a0 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.941+0000 7fc8bc565740 10 client.15291 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.941+0000 7fc8bc565740 1 -- 172.21.15.17:0/2117161896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f63c682d0 con 0x557f63ff42a0 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.941+0000 7fc892ffd640 1 -- 172.21.15.17:0/2117161896 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8a4004b40 con 0x557f63ff42a0 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.941+0000 7fc892ffd640 1 -- 172.21.15.17:0/2117161896 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc8a404b070 con 0x557f63ff42a0 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.941+0000 7fc8bc565740 10 client.15291 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.941+0000 7fc8bc565740 10 client.15291 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.941+0000 7fc8bc565740 20 client.15291 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.941+0000 7fc8bc565740 1 --2- 172.21.15.17:0/2117161896 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x557f64004fd0 0x557f64009450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.941+0000 7fc8bc565740 4 client.15291 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.941+0000 7fc8bc565740 1 -- 172.21.15.17:0/2117161896 --> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f63f61160 con 0x557f64004fd0 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.941+0000 7fc8b2ffd640 1 --2- 172.21.15.17:0/2117161896 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x557f64004fd0 0x557f64009450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc892ffd640 10 client.15291 ms_handle_connect on v2:172.21.15.19:6838/2102503456 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc8b2ffd640 1 --2- 172.21.15.17:0/2117161896 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x557f64004fd0 0x557f64009450 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=343831f5d9d00bf9 server_cookie=a0e1a06b7651d224 in_seq=0 out_seq=0 2024-03-24T03:00:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc892ffd640 1 -- 172.21.15.17:0/2117161896 <== mds.1 v2:172.21.15.19:6838/2102503456 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557f63f61160 con 0x557f64004fd0 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc892ffd640 10 client.15291 handle_command_reply: tid=0 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc8bc565740 1 client.15291 shutdown 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc8bc565740 1 -- 172.21.15.17:0/2117161896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 msgr2=0x557f64000170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc8bc565740 1 --2- 172.21.15.17:0/2117161896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f64000170 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4037040 tx=0x7fc8a4004470 comp rx=0 tx=0).stop 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc8bc565740 1 -- 172.21.15.17:0/2117161896 shutdown_connections 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc8bc565740 1 --2- 172.21.15.17:0/2117161896 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x557f64004fd0 0x557f64009450 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc8bc565740 1 --2- 172.21.15.17:0/2117161896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f64000170 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc8bc565740 1 --2- 172.21.15.17:0/2117161896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fef130 0x557f63ff91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc892ffd640 0 client.15291 ms_handle_reset on v2:172.21.15.19:6838/2102503456 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc8bc565740 1 --2- 172.21.15.17:0/2117161896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fee740 0x557f63ff8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.942+0000 7fc8bc565740 1 -- 172.21.15.17:0/2117161896 >> 172.21.15.17:0/2117161896 conn(0x557f63fb6a60 msgr2=0x557f63fe92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.943+0000 7fc8bc565740 1 -- 172.21.15.17:0/2117161896 shutdown_connections 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.943+0000 7fc8bc565740 1 -- 172.21.15.17:0/2117161896 wait complete. 2024-03-24T03:00:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.943+0000 7fc8bc565740 20 client.15291 trim_cache size 0 max 16384 2024-03-24T03:00:23.955 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:23 smithi017 ceph-mon[31926]: pgmap v347: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-24T03:00:23.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:23 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2746293728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:23.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:23 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3642083499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.959+0000 7fc8bc565740 1 Processor -- start 2024-03-24T03:00:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.959+0000 7fc8bc565740 1 -- start start 2024-03-24T03:00:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8bc565740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fee740 0x557f63ff8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8bc565740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fef130 0x557f6400d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8bc565740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f6400d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8bc565740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f640047f0 con 0x557f63ff42a0 2024-03-24T03:00:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8bc565740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f64004e50 con 0x557f63fef130 2024-03-24T03:00:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8bc565740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f640054b0 con 0x557f63fee740 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f6400d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f6400d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35790/0 (socket says 172.21.15.17:35790) 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8b37fe640 1 -- 172.21.15.17:0/642766474 learned_addr learned my addr 172.21.15.17:0/642766474 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8b27fc640 1 --2- 172.21.15.17:0/642766474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fef130 0x557f6400d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8b2ffd640 1 --2- 172.21.15.17:0/642766474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fee740 0x557f63ff8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8b37fe640 1 -- 172.21.15.17:0/642766474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fee740 msgr2=0x557f63ff8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/642766474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fee740 0x557f63ff8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8b37fe640 1 -- 172.21.15.17:0/642766474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fef130 msgr2=0x557f6400d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/642766474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fef130 0x557f6400d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8b37fe640 1 -- 172.21.15.17:0/642766474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f63d326b0 con 0x557f63ff42a0 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/642766474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f6400d9b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4005e30 tx=0x7fc8a4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6d364064550ffb server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8a97fa640 1 -- 172.21.15.17:0/642766474 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a405a070 con 0x557f63ff42a0 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8a97fa640 1 -- 172.21.15.17:0/642766474 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8a4034920 con 0x557f63ff42a0 2024-03-24T03:00:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.960+0000 7fc8a97fa640 1 -- 172.21.15.17:0/642766474 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4039050 con 0x557f63ff42a0 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.961+0000 7fc8bc565740 1 -- 172.21.15.17:0/642766474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 msgr2=0x557f6400d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.961+0000 7fc8bc565740 1 --2- 172.21.15.17:0/642766474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f6400d9b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4005e30 tx=0x7fc8a4005b50 comp rx=0 tx=0).stop 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.961+0000 7fc8bc565740 1 -- 172.21.15.17:0/642766474 shutdown_connections 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.961+0000 7fc8bc565740 1 --2- 172.21.15.17:0/642766474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f6400d9b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.961+0000 7fc8bc565740 1 --2- 172.21.15.17:0/642766474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fef130 0x557f6400d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.961+0000 7fc8bc565740 1 --2- 172.21.15.17:0/642766474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fee740 0x557f63ff8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.961+0000 7fc8bc565740 1 -- 172.21.15.17:0/642766474 >> 172.21.15.17:0/642766474 conn(0x557f63fb6a60 msgr2=0x557f63ff8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.961+0000 7fc8bc565740 1 -- 172.21.15.17:0/642766474 shutdown_connections 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.961+0000 7fc8bc565740 1 -- 172.21.15.17:0/642766474 wait complete. 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.961+0000 7fc8bc565740 1 -- start start 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.961+0000 7fc8bc565740 1 Processor -- start 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8bc565740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fee740 0x557f6405e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8bc565740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fef130 0x557f64060ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8bc565740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f640630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8bc565740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f64006170 con 0x557f63ff42a0 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8bc565740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f64005b10 con 0x557f63fef130 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8bc565740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f640067d0 con 0x557f63fee740 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8bc565740 10 client.0 _ll_register_callbacks cb 0x7fc8a404f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f640630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f640630f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35798/0 (socket says 172.21.15.17:35798) 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8b27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fef130 0x557f64060ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8b37fe640 1 -- 172.21.15.17:0/3041093442 learned_addr learned my addr 172.21.15.17:0/3041093442 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8b2ffd640 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fee740 0x557f6405e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8b27fc640 1 -- 172.21.15.17:0/3041093442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fee740 msgr2=0x557f6405e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8b27fc640 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fee740 0x557f6405e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8b27fc640 1 -- 172.21.15.17:0/3041093442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 msgr2=0x557f640630f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8b27fc640 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f640630f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8b27fc640 1 -- 172.21.15.17:0/3041093442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f64063920 con 0x557f63fef130 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8b37fe640 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f640630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8b27fc640 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fef130 0x557f64060ab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x557f63fefb50 tx=0x7fc89c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62066b7d9eddf7a server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc892ffd640 1 -- 172.21.15.17:0/3041093442 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c023060 con 0x557f63fef130 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc8bc565740 1 -- 172.21.15.17:0/3041093442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8a4034920 con 0x557f63fef130 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.962+0000 7fc892ffd640 1 -- 172.21.15.17:0/3041093442 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fc89c004050 con 0x557f63fef130 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc892ffd640 1 -- 172.21.15.17:0/3041093442 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc89c004250 con 0x557f63fef130 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc8bc565740 10 client.25067 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc8bc565740 1 -- 172.21.15.17:0/3041093442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f64064d40 con 0x557f63fef130 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc892ffd640 1 -- 172.21.15.17:0/3041093442 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8a4034920 con 0x557f63fef130 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc892ffd640 1 -- 172.21.15.17:0/3041093442 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc89c004aa0 con 0x557f63fef130 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc8bc565740 10 client.25067 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc8bc565740 10 client.25067 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc8bc565740 20 client.25067 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:23.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc8bc565740 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x557f64069380 0x557f64065030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc8bc565740 4 client.25067 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc8bc565740 1 -- 172.21.15.17:0/3041093442 --> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557f63f5b4b0 con 0x557f64069380 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc8b2ffd640 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x557f64069380 0x557f64065030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc892ffd640 10 client.25067 ms_handle_connect on v2:172.21.15.19:6838/2102503456 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.963+0000 7fc8b2ffd640 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x557f64069380 0x557f64065030 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85490dade92010e9 server_cookie=43382b9cac8efc49 in_seq=0 out_seq=0 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc892ffd640 1 -- 172.21.15.17:0/3041093442 <== mds.1 v2:172.21.15.19:6838/2102503456 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x557f63f5b4b0 con 0x557f64069380 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc892ffd640 10 client.25067 handle_command_reply: tid=0 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8bc565740 1 client.25067 shutdown 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8bc565740 1 -- 172.21.15.17:0/3041093442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fef130 msgr2=0x557f64060ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8bc565740 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fef130 0x557f64060ab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x557f63fefb50 tx=0x7fc89c00f790 comp rx=0 tx=0).stop 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8bc565740 1 -- 172.21.15.17:0/3041093442 shutdown_connections 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8bc565740 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x557f64069380 0x557f64065030 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8bc565740 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f63ff42a0 0x557f640630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8bc565740 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f63fef130 0x557f64060ab0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc892ffd640 0 client.25067 ms_handle_reset on v2:172.21.15.19:6838/2102503456 2024-03-24T03:00:23.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8bc565740 1 --2- 172.21.15.17:0/3041093442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f63fee740 0x557f6405e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8bc565740 1 -- 172.21.15.17:0/3041093442 >> 172.21.15.17:0/3041093442 conn(0x557f63fb6a60 msgr2=0x557f63fed2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8bc565740 1 -- 172.21.15.17:0/3041093442 shutdown_connections 2024-03-24T03:00:23.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8bc565740 1 -- 172.21.15.17:0/3041093442 wait complete. 2024-03-24T03:00:23.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8bc565740 20 client.25067 trim_cache size 0 max 16384 2024-03-24T03:00:23.971 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:00:23.971 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T03:00:23.971 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 1, 2024-03-24T03:00:23.971 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24521, 2024-03-24T03:00:23.971 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-24T03:00:23.972 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-24T03:00:23.972 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-24T03:00:23.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 88.923853270999999, 2024-03-24T03:00:23.972 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 28, 2024-03-24T03:00:23.972 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 91, 2024-03-24T03:00:23.972 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-03-24T03:00:23.972 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 112.981740844 2024-03-24T03:00:23.972 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:00:23.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8b9977640 1 -- 172.21.15.17:0/1849782475 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8a00a2800 msgr2=0x7fc8a00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8b9977640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8a00a2800 0x7fc8a00a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009430 tx=0x7fc8a4042000 comp rx=0 tx=0).stop 2024-03-24T03:00:23.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8b9977640 1 -- 172.21.15.17:0/1849782475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b41021c0 msgr2=0x7fc8b41979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:23.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8b9977640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b41021c0 0x7fc8b41979c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00e9f0 tx=0x7fc8ac00eeb0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8b9977640 1 -- 172.21.15.17:0/1849782475 shutdown_connections 2024-03-24T03:00:23.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8b9977640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8a00a2800 0x7fc8a00a4cc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8b9977640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8b4107420 0x7fc8b419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8b9977640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b4104af0 0x7fc8b4197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8b9977640 1 --2- 172.21.15.17:0/1849782475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b41021c0 0x7fc8b41979c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:23.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.964+0000 7fc8b9977640 1 -- 172.21.15.17:0/1849782475 >> 172.21.15.17:0/1849782475 conn(0x7fc8b40fbb30 msgr2=0x7fc8b4102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:23.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.965+0000 7fc8b9977640 1 -- 172.21.15.17:0/1849782475 shutdown_connections 2024-03-24T03:00:23.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:23.965+0000 7fc8b9977640 1 -- 172.21.15.17:0/1849782475 wait complete. 2024-03-24T03:00:23.975 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-24T03:00:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:23 smithi019 ceph-mon[35944]: pgmap v347: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-24T03:00:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:23 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2746293728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:23 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3642083499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:24.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.079+0000 7fce6971d640 1 -- 172.21.15.17:0/1090770398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce64101790 msgr2=0x7fce64101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:24.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.079+0000 7fce6971d640 1 --2- 172.21.15.17:0/1090770398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce64101790 0x7fce64101c10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fce4c009900 tx=0x7fce4c02bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:24.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.079+0000 7fce6971d640 1 -- 172.21.15.17:0/1090770398 shutdown_connections 2024-03-24T03:00:24.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.079+0000 7fce6971d640 1 --2- 172.21.15.17:0/1090770398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce64102150 0x7fce641089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.079+0000 7fce6971d640 1 --2- 172.21.15.17:0/1090770398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce64101790 0x7fce64101c10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.079+0000 7fce6971d640 1 --2- 172.21.15.17:0/1090770398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce641005a0 0x7fce641009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.079+0000 7fce6971d640 1 -- 172.21.15.17:0/1090770398 >> 172.21.15.17:0/1090770398 conn(0x7fce640fbb30 msgr2=0x7fce640fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:24.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.079+0000 7fce6971d640 1 -- 172.21.15.17:0/1090770398 shutdown_connections 2024-03-24T03:00:24.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.079+0000 7fce6971d640 1 -- 172.21.15.17:0/1090770398 wait complete. 2024-03-24T03:00:24.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce6971d640 1 Processor -- start 2024-03-24T03:00:24.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce6971d640 1 -- start start 2024-03-24T03:00:24.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce6971d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce641005a0 0x7fce64069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce6971d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce64101790 0x7fce6419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce6971d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce64102150 0x7fce6419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce6971d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6410c270 con 0x7fce641005a0 2024-03-24T03:00:24.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce6971d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6410c0f0 con 0x7fce64102150 2024-03-24T03:00:24.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce6971d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6410c3f0 con 0x7fce64101790 2024-03-24T03:00:24.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce62ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce641005a0 0x7fce64069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce62ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce641005a0 0x7fce64069390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35802/0 (socket says 172.21.15.17:35802) 2024-03-24T03:00:24.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce62ffd640 1 -- 172.21.15.17:0/951593209 learned_addr learned my addr 172.21.15.17:0/951593209 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:24.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce637fe640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce64102150 0x7fce6419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.080+0000 7fce627fc640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce64101790 0x7fce6419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.081+0000 7fce62ffd640 1 -- 172.21.15.17:0/951593209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce64101790 msgr2=0x7fce6419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:24.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.081+0000 7fce62ffd640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce64101790 0x7fce6419c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.081+0000 7fce62ffd640 1 -- 172.21.15.17:0/951593209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce64102150 msgr2=0x7fce6419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:24.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.081+0000 7fce62ffd640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce64102150 0x7fce6419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.081+0000 7fce62ffd640 1 -- 172.21.15.17:0/951593209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6419f610 con 0x7fce641005a0 2024-03-24T03:00:24.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.081+0000 7fce637fe640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce64102150 0x7fce6419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:24.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.081+0000 7fce62ffd640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce641005a0 0x7fce64069390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00e9c0 tx=0x7fce5c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:24.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.081+0000 7fce5bfff640 1 -- 172.21.15.17:0/951593209 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c00cde0 con 0x7fce641005a0 2024-03-24T03:00:24.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.081+0000 7fce5bfff640 1 -- 172.21.15.17:0/951593209 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce5c010910 con 0x7fce641005a0 2024-03-24T03:00:24.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.081+0000 7fce6971d640 1 -- 172.21.15.17:0/951593209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6419f900 con 0x7fce641005a0 2024-03-24T03:00:24.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.081+0000 7fce5bfff640 1 -- 172.21.15.17:0/951593209 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c018a00 con 0x7fce641005a0 2024-03-24T03:00:24.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.081+0000 7fce6971d640 1 -- 172.21.15.17:0/951593209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6419fe20 con 0x7fce641005a0 2024-03-24T03:00:24.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.083+0000 7fce5bfff640 1 -- 172.21.15.17:0/951593209 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fce5c01f020 con 0x7fce641005a0 2024-03-24T03:00:24.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.083+0000 7fce6c3df740 1 Processor -- start 2024-03-24T03:00:24.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.083+0000 7fce6c3df740 1 -- start start 2024-03-24T03:00:24.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce6c3df740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc3740 0x55f916dc3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce6c3df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dc8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce5bfff640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fce500a2800 0x7fce500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce6c3df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc92a0 0x55f916dcb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce6c3df740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f916d28c20 con 0x55f916dc3740 2024-03-24T03:00:24.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce6c3df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f916cd1a40 con 0x55f916dc92a0 2024-03-24T03:00:24.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce6c3df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f916d24cc0 con 0x55f916dc4130 2024-03-24T03:00:24.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce62ffd640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fce500a2800 0x7fce500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce627fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dc8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce627fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dc8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:57492/0 (socket says 172.21.15.17:57492) 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce627fc640 1 -- 172.21.15.17:0/2638387445 learned_addr learned my addr 172.21.15.17:0/2638387445 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce637fe640 1 --2- 172.21.15.17:0/2638387445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc3740 0x55f916dc3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce5bfff640 1 -- 172.21.15.17:0/951593209 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fce5c0cef70 con 0x7fce641005a0 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce637fe640 1 --2- 172.21.15.17:0/2638387445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc92a0 0x55f916dcb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce627fc640 1 -- 172.21.15.17:0/2638387445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc92a0 msgr2=0x55f916dcb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce627fc640 1 --2- 172.21.15.17:0/2638387445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc92a0 0x55f916dcb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce627fc640 1 -- 172.21.15.17:0/2638387445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc3740 msgr2=0x55f916dc3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce627fc640 1 --2- 172.21.15.17:0/2638387445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc3740 0x55f916dc3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce627fc640 1 -- 172.21.15.17:0/2638387445 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f916cf3f90 con 0x55f916dc4130 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce62ffd640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fce500a2800 0x7fce500a4cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0979e0 tx=0x7fce5c08d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:24.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.084+0000 7fce627fc640 1 --2- 172.21.15.17:0/2638387445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dc8c30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce4c000c00 tx=0x7fce4c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1eaa22f7ed4fa83 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.085+0000 7fce597fa640 1 -- 172.21.15.17:0/2638387445 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c045070 con 0x55f916dc4130 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.085+0000 7fce597fa640 1 -- 172.21.15.17:0/2638387445 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce4c004040 con 0x55f916dc4130 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.085+0000 7fce597fa640 1 -- 172.21.15.17:0/2638387445 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c004cb0 con 0x55f916dc4130 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.085+0000 7fce6c3df740 1 -- 172.21.15.17:0/2638387445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 msgr2=0x55f916dc8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.085+0000 7fce6c3df740 1 --2- 172.21.15.17:0/2638387445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dc8c30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce4c000c00 tx=0x7fce4c009560 comp rx=0 tx=0).stop 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.085+0000 7fce6c3df740 1 -- 172.21.15.17:0/2638387445 shutdown_connections 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.085+0000 7fce6c3df740 1 --2- 172.21.15.17:0/2638387445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc92a0 0x55f916dcb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.085+0000 7fce6c3df740 1 --2- 172.21.15.17:0/2638387445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dc8c30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.085+0000 7fce6c3df740 1 --2- 172.21.15.17:0/2638387445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc3740 0x55f916dc3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.085+0000 7fce6c3df740 1 -- 172.21.15.17:0/2638387445 >> 172.21.15.17:0/2638387445 conn(0x55f916d8ba60 msgr2=0x55f916dbf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.085+0000 7fce6c3df740 1 -- 172.21.15.17:0/2638387445 shutdown_connections 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.085+0000 7fce6c3df740 1 -- 172.21.15.17:0/2638387445 wait complete. 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce6c3df740 1 -- start start 2024-03-24T03:00:24.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce6c3df740 1 Processor -- start 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce6c3df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc3740 0x55f916dcdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce6c3df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc4130 0x55f916dce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce6c3df740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 0x55f916dd5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce6c3df740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f916dcc1f0 con 0x55f916dc92a0 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce6c3df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f916ce4e90 con 0x55f916dc4130 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce6c3df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f916dcc370 con 0x55f916dc3740 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce6c3df740 10 client.0 _ll_register_callbacks cb 0x55f916a3cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 0x55f916dd5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce627fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc4130 0x55f916dce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 0x55f916dd5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35814/0 (socket says 172.21.15.17:35814) 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce637fe640 1 -- 172.21.15.17:0/4125151971 learned_addr learned my addr 172.21.15.17:0/4125151971 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce637fe640 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc3740 0x55f916dcdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce627fc640 1 -- 172.21.15.17:0/4125151971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc3740 msgr2=0x55f916dcdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce627fc640 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc3740 0x55f916dcdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce627fc640 1 -- 172.21.15.17:0/4125151971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 msgr2=0x55f916dd5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce627fc640 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 0x55f916dd5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce627fc640 1 -- 172.21.15.17:0/4125151971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f916cf19d0 con 0x55f916dc4130 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.086+0000 7fce637fe640 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 0x55f916dd5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce627fc640 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc4130 0x55f916dce100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55f916dc0000 tx=0x7fce4c0044a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d2dbd7dd731a699 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce4a7fc640 1 -- 172.21.15.17:0/4125151971 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c04e070 con 0x55f916dc4130 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce4a7fc640 1 -- 172.21.15.17:0/4125151971 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fce4c004eb0 con 0x55f916dc4130 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce6c3df740 1 -- 172.21.15.17:0/4125151971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce4c004040 con 0x55f916dc4130 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce4a7fc640 1 -- 172.21.15.17:0/4125151971 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce4c059050 con 0x55f916dc4130 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce6c3df740 10 client.25079 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:24.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce6c3df740 1 -- 172.21.15.17:0/4125151971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f916a3d2d0 con 0x55f916dc4130 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce4a7fc640 1 -- 172.21.15.17:0/4125151971 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce4c004040 con 0x55f916dc4130 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce4a7fc640 1 -- 172.21.15.17:0/4125151971 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fce4c003980 con 0x55f916dc4130 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce6c3df740 10 client.25079 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce6c3df740 10 client.25079 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce6c3df740 20 client.25079 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce6c3df740 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55f916dd92a0 0x55f916dde340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce6c3df740 4 client.25079 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.087+0000 7fce6c3df740 1 -- 172.21.15.17:0/4125151971 --> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f916d36160 con 0x55f916dd92a0 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.088+0000 7fce637fe640 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55f916dd92a0 0x55f916dde340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.088+0000 7fce4a7fc640 10 client.25079 ms_handle_connect on v2:172.21.15.188:6838/81409664 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.088+0000 7fce637fe640 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55f916dd92a0 0x55f916dde340 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ba2cb1ccafff1fa server_cookie=d465bc4f74c887d6 in_seq=0 out_seq=0 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce4a7fc640 1 -- 172.21.15.17:0/4125151971 <== mds.0 v2:172.21.15.188:6838/81409664 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f916d36160 con 0x55f916dd92a0 2024-03-24T03:00:24.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce4a7fc640 10 client.25079 handle_command_reply: tid=0 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce6c3df740 1 client.25079 shutdown 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce6c3df740 1 -- 172.21.15.17:0/4125151971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc4130 msgr2=0x55f916dce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce6c3df740 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc4130 0x55f916dce100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55f916dc0000 tx=0x7fce4c0044a0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce6c3df740 1 -- 172.21.15.17:0/4125151971 shutdown_connections 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce6c3df740 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55f916dd92a0 0x55f916dde340 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce6c3df740 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 0x55f916dd5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce4a7fc640 0 client.25079 ms_handle_reset on v2:172.21.15.188:6838/81409664 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce6c3df740 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc4130 0x55f916dce100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce6c3df740 1 --2- 172.21.15.17:0/4125151971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc3740 0x55f916dcdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce6c3df740 1 -- 172.21.15.17:0/4125151971 >> 172.21.15.17:0/4125151971 conn(0x55f916d8ba60 msgr2=0x55f916dbe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce6c3df740 1 -- 172.21.15.17:0/4125151971 shutdown_connections 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce6c3df740 1 -- 172.21.15.17:0/4125151971 wait complete. 2024-03-24T03:00:24.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.089+0000 7fce6c3df740 20 client.25079 trim_cache size 0 max 16384 2024-03-24T03:00:24.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.105+0000 7fce6c3df740 1 Processor -- start 2024-03-24T03:00:24.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce6c3df740 1 -- start start 2024-03-24T03:00:24.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce6c3df740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc3740 0x55f916dbe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce6c3df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dd5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce6c3df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc92a0 0x55f916dcd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce6c3df740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f916dd96e0 con 0x55f916dc3740 2024-03-24T03:00:24.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce6c3df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f916dd9d40 con 0x55f916dc92a0 2024-03-24T03:00:24.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce6c3df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f916dda3a0 con 0x55f916dc4130 2024-03-24T03:00:24.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc3740 0x55f916dbe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc3740 0x55f916dbe170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35822/0 (socket says 172.21.15.17:35822) 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce637fe640 1 -- 172.21.15.17:0/1850474016 learned_addr learned my addr 172.21.15.17:0/1850474016 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce627fc640 1 --2- 172.21.15.17:0/1850474016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dd5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce637fe640 1 --2- 172.21.15.17:0/1850474016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc92a0 0x55f916dcd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce637fe640 1 -- 172.21.15.17:0/1850474016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 msgr2=0x55f916dd5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce637fe640 1 --2- 172.21.15.17:0/1850474016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dd5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce637fe640 1 -- 172.21.15.17:0/1850474016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc92a0 msgr2=0x55f916dcd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce637fe640 1 --2- 172.21.15.17:0/1850474016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc92a0 0x55f916dcd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce637fe640 1 -- 172.21.15.17:0/1850474016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f916b076b0 con 0x55f916dc3740 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.106+0000 7fce637fe640 1 --2- 172.21.15.17:0/1850474016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc3740 0x55f916dbe170 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fce54005fd0 tx=0x7fce54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb7b4b34494e421 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.107+0000 7fce58ff9640 1 -- 172.21.15.17:0/1850474016 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54016030 con 0x55f916dc3740 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.107+0000 7fce58ff9640 1 -- 172.21.15.17:0/1850474016 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce54003e50 con 0x55f916dc3740 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.107+0000 7fce58ff9640 1 -- 172.21.15.17:0/1850474016 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54004050 con 0x55f916dc3740 2024-03-24T03:00:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.107+0000 7fce6c3df740 1 -- 172.21.15.17:0/1850474016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc3740 msgr2=0x55f916dbe170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.107+0000 7fce6c3df740 1 --2- 172.21.15.17:0/1850474016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc3740 0x55f916dbe170 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fce54005fd0 tx=0x7fce54004a40 comp rx=0 tx=0).stop 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.107+0000 7fce6c3df740 1 -- 172.21.15.17:0/1850474016 shutdown_connections 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.107+0000 7fce6c3df740 1 --2- 172.21.15.17:0/1850474016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc92a0 0x55f916dcd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.107+0000 7fce6c3df740 1 --2- 172.21.15.17:0/1850474016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dd5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.107+0000 7fce6c3df740 1 --2- 172.21.15.17:0/1850474016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc3740 0x55f916dbe170 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.107+0000 7fce6c3df740 1 -- 172.21.15.17:0/1850474016 >> 172.21.15.17:0/1850474016 conn(0x55f916d8ba60 msgr2=0x55f916dbda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.107+0000 7fce6c3df740 1 -- 172.21.15.17:0/1850474016 shutdown_connections 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.107+0000 7fce6c3df740 1 -- 172.21.15.17:0/1850474016 wait complete. 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce6c3df740 1 -- start start 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce6c3df740 1 Processor -- start 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce6c3df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc3740 0x55f916dd1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce6c3df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dd1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce6c3df740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 0x55f916dd4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce6c3df740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f916ddb060 con 0x55f916dc92a0 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce6c3df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f916ddaa00 con 0x55f916dc3740 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce6c3df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f916ddb6c0 con 0x55f916dc4130 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce6c3df740 10 client.0 _ll_register_callbacks cb 0x7fce540115e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 0x55f916dd4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 0x55f916dd4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35830/0 (socket says 172.21.15.17:35830) 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce627fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dd1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce637fe640 1 -- 172.21.15.17:0/3258084572 learned_addr learned my addr 172.21.15.17:0/3258084572 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce637fe640 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc3740 0x55f916dd1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce637fe640 1 -- 172.21.15.17:0/3258084572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 msgr2=0x55f916dd1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce637fe640 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dd1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce637fe640 1 -- 172.21.15.17:0/3258084572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc3740 msgr2=0x55f916dd1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce637fe640 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc3740 0x55f916dd1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce637fe640 1 -- 172.21.15.17:0/3258084572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f916dd4b50 con 0x55f916dc92a0 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.108+0000 7fce627fc640 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dd1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:24.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce637fe640 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 0x55f916dd4560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fce5400fd40 tx=0x7fce54013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a2b77b58eeff29 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce4a7fc640 1 -- 172.21.15.17:0/3258084572 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54016040 con 0x55f916dc92a0 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce4a7fc640 1 -- 172.21.15.17:0/3258084572 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fce54003970 con 0x55f916dc92a0 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce6c3df740 1 -- 172.21.15.17:0/3258084572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce54003e50 con 0x55f916dc92a0 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce4a7fc640 1 -- 172.21.15.17:0/3258084572 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce54042050 con 0x55f916dc92a0 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce6c3df740 10 client.15315 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce6c3df740 1 -- 172.21.15.17:0/3258084572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f916e35c80 con 0x55f916dc92a0 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce4a7fc640 1 -- 172.21.15.17:0/3258084572 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce54003e50 con 0x55f916dc92a0 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce4a7fc640 1 -- 172.21.15.17:0/3258084572 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fce54031040 con 0x55f916dc92a0 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce6c3df740 10 client.15315 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce6c3df740 10 client.15315 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce6c3df740 20 client.15315 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce6c3df740 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55f916e3a080 0x55f916e35fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce6c3df740 4 client.15315 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-24T03:00:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce6c3df740 1 -- 172.21.15.17:0/3258084572 --> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f916d304b0 con 0x55f916e3a080 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.109+0000 7fce627fc640 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55f916e3a080 0x55f916e35fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce4a7fc640 10 client.15315 ms_handle_connect on v2:172.21.15.188:6838/81409664 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce627fc640 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55f916e3a080 0x55f916e35fc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914443a5ec5be089 server_cookie=55a9ff2240b3c5f1 in_seq=0 out_seq=0 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce4a7fc640 1 -- 172.21.15.17:0/3258084572 <== mds.0 v2:172.21.15.188:6838/81409664 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55f916d304b0 con 0x55f916e3a080 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce4a7fc640 10 client.15315 handle_command_reply: tid=0 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce6c3df740 1 client.15315 shutdown 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce6c3df740 1 -- 172.21.15.17:0/3258084572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 msgr2=0x55f916dd4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce6c3df740 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 0x55f916dd4560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fce5400fd40 tx=0x7fce54013040 comp rx=0 tx=0).stop 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce6c3df740 1 -- 172.21.15.17:0/3258084572 shutdown_connections 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce6c3df740 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55f916e3a080 0x55f916e35fc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce6c3df740 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f916dc92a0 0x55f916dd4560 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce6c3df740 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f916dc4130 0x55f916dd1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce6c3df740 1 --2- 172.21.15.17:0/3258084572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f916dc3740 0x55f916dd1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce6c3df740 1 -- 172.21.15.17:0/3258084572 >> 172.21.15.17:0/3258084572 conn(0x55f916d8ba60 msgr2=0x55f916dc2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce4a7fc640 0 client.15315 ms_handle_reset on v2:172.21.15.188:6838/81409664 2024-03-24T03:00:24.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce6c3df740 1 -- 172.21.15.17:0/3258084572 shutdown_connections 2024-03-24T03:00:24.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce6c3df740 1 -- 172.21.15.17:0/3258084572 wait complete. 2024-03-24T03:00:24.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.110+0000 7fce6c3df740 20 client.15315 trim_cache size 0 max 16384 2024-03-24T03:00:24.118 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:00:24.118 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T03:00:24.118 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-24T03:00:24.118 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24529, 2024-03-24T03:00:24.118 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-24T03:00:24.118 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-24T03:00:24.118 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-24T03:00:24.118 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 106.737242759, 2024-03-24T03:00:24.119 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 28, 2024-03-24T03:00:24.119 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 91, 2024-03-24T03:00:24.119 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-03-24T03:00:24.119 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 121.710691658 2024-03-24T03:00:24.119 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:00:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.111+0000 7fce6971d640 1 -- 172.21.15.17:0/951593209 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fce500a2800 msgr2=0x7fce500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.111+0000 7fce6971d640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fce500a2800 0x7fce500a4cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0979e0 tx=0x7fce5c08d040 comp rx=0 tx=0).stop 2024-03-24T03:00:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.111+0000 7fce6971d640 1 -- 172.21.15.17:0/951593209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce641005a0 msgr2=0x7fce64069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.111+0000 7fce6971d640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce641005a0 0x7fce64069390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00e9c0 tx=0x7fce5c00ee80 comp rx=0 tx=0).stop 2024-03-24T03:00:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.111+0000 7fce6971d640 1 -- 172.21.15.17:0/951593209 shutdown_connections 2024-03-24T03:00:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.111+0000 7fce6971d640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fce500a2800 0x7fce500a4cc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.111+0000 7fce6971d640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce64102150 0x7fce6419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.111+0000 7fce6971d640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce64101790 0x7fce6419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.111+0000 7fce6971d640 1 --2- 172.21.15.17:0/951593209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce641005a0 0x7fce64069390 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.111+0000 7fce6971d640 1 -- 172.21.15.17:0/951593209 >> 172.21.15.17:0/951593209 conn(0x7fce640fbb30 msgr2=0x7fce64104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.111+0000 7fce6971d640 1 -- 172.21.15.17:0/951593209 shutdown_connections 2024-03-24T03:00:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:24.111+0000 7fce6971d640 1 -- 172.21.15.17:0/951593209 wait complete. 2024-03-24T03:00:24.121 INFO:tasks.cephadm:Setup complete, yielding 2024-03-24T03:00:24.122 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T03:00:24.132 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2024-03-24T03:00:24.132 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-24T03:00:24.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:23 smithi188 ceph-mon[36206]: pgmap v347: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-24T03:00:24.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:23 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2746293728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:24.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:23 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3642083499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:24.374 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:00:26.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:25 smithi019 ceph-mon[35944]: pgmap v348: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T03:00:26.118 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:25 smithi017 ceph-mon[31926]: pgmap v348: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T03:00:26.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.226+0000 7fc841da6700 1 -- 172.21.15.17:0/3448818865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c103e50 msgr2=0x7fc83c10a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:26.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.226+0000 7fc841da6700 1 --2- 172.21.15.17:0/3448818865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c103e50 0x7fc83c10a700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc834009990 tx=0x7fc834009ca0 comp rx=0 tx=0).stop 2024-03-24T03:00:26.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.226+0000 7fc841da6700 1 -- 172.21.15.17:0/3448818865 shutdown_connections 2024-03-24T03:00:26.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.226+0000 7fc841da6700 1 --2- 172.21.15.17:0/3448818865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c103e50 0x7fc83c10a700 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:26.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.226+0000 7fc841da6700 1 --2- 172.21.15.17:0/3448818865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc83c1004d0 0x7fc83c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.226+0000 7fc841da6700 1 --2- 172.21.15.17:0/3448818865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c1034b0 0x7fc83c103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.226+0000 7fc841da6700 1 -- 172.21.15.17:0/3448818865 >> 172.21.15.17:0/3448818865 conn(0x7fc83c0fbe30 msgr2=0x7fc83c0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.226+0000 7fc841da6700 1 -- 172.21.15.17:0/3448818865 shutdown_connections 2024-03-24T03:00:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.226+0000 7fc841da6700 1 -- 172.21.15.17:0/3448818865 wait complete. 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.226+0000 7fc841da6700 1 Processor -- start 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc841da6700 1 -- start start 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc841da6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c1004d0 0x7fc83c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc841da6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc83c1034b0 0x7fc83c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc841da6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c103e50 0x7fc83c19f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc841da6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c10dde0 con 0x7fc83c1004d0 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc841da6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c10dc60 con 0x7fc83c103e50 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc841da6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c10df60 con 0x7fc83c1034b0 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc83b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c1004d0 0x7fc83c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc83b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c1004d0 0x7fc83c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35862/0 (socket says 172.21.15.17:35862) 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc83b7fe700 1 -- 172.21.15.17:0/3464685570 learned_addr learned my addr 172.21.15.17:0/3464685570 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc83affd700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc83c1034b0 0x7fc83c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc83bfff700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c103e50 0x7fc83c19f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc83b7fe700 1 -- 172.21.15.17:0/3464685570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc83c1034b0 msgr2=0x7fc83c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc83b7fe700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc83c1034b0 0x7fc83c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc83b7fe700 1 -- 172.21.15.17:0/3464685570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c103e50 msgr2=0x7fc83c19f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc83b7fe700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c103e50 0x7fc83c19f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.227+0000 7fc83b7fe700 1 -- 172.21.15.17:0/3464685570 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c1a0030 con 0x7fc83c1004d0 2024-03-24T03:00:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.228+0000 7fc83b7fe700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c1004d0 0x7fc83c1983d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009f40 tx=0x7fc82c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.228+0000 7fc838ff9700 1 -- 172.21.15.17:0/3464685570 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0092e0 con 0x7fc83c1004d0 2024-03-24T03:00:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.228+0000 7fc841da6700 1 -- 172.21.15.17:0/3464685570 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83c1a02c0 con 0x7fc83c1004d0 2024-03-24T03:00:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.228+0000 7fc838ff9700 1 -- 172.21.15.17:0/3464685570 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc82c00c9c0 con 0x7fc83c1004d0 2024-03-24T03:00:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.228+0000 7fc838ff9700 1 -- 172.21.15.17:0/3464685570 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c004580 con 0x7fc83c1004d0 2024-03-24T03:00:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.228+0000 7fc841da6700 1 -- 172.21.15.17:0/3464685570 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83c1a0850 con 0x7fc83c1004d0 2024-03-24T03:00:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.229+0000 7fc841da6700 1 -- 172.21.15.17:0/3464685570 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83c068e50 con 0x7fc83c1004d0 2024-03-24T03:00:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.230+0000 7fc838ff9700 1 -- 172.21.15.17:0/3464685570 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc82c01b030 con 0x7fc83c1004d0 2024-03-24T03:00:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.230+0000 7fc838ff9700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8240a2890 0x7fc8240a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.231+0000 7fc83affd700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8240a2890 0x7fc8240a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:26.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.231+0000 7fc838ff9700 1 -- 172.21.15.17:0/3464685570 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7fc82c0cecc0 con 0x7fc83c1004d0 2024-03-24T03:00:26.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.231+0000 7fc83affd700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8240a2890 0x7fc8240a4d50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc830009c60 tx=0x7fc830009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:26.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.235+0000 7fc838ff9700 1 -- 172.21.15.17:0/3464685570 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc82c022020 con 0x7fc83c1004d0 2024-03-24T03:00:26.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:25 smithi188 ceph-mon[36206]: pgmap v348: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T03:00:26.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.458+0000 7fc841da6700 1 -- 172.21.15.17:0/3464685570 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc83c0625d0 con 0x7fc8240a2890 2024-03-24T03:00:26.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.459+0000 7fc838ff9700 1 -- 172.21.15.17:0/3464685570 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc83c0625d0 con 0x7fc8240a2890 2024-03-24T03:00:26.458 INFO:teuthology.orchestra.run.smithi017.stdout:Backend: cephadm 2024-03-24T03:00:26.458 INFO:teuthology.orchestra.run.smithi017.stdout:Available: Yes 2024-03-24T03:00:26.458 INFO:teuthology.orchestra.run.smithi017.stdout:Paused: No 2024-03-24T03:00:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.461+0000 7fc841da6700 1 -- 172.21.15.17:0/3464685570 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8240a2890 msgr2=0x7fc8240a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.461+0000 7fc841da6700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8240a2890 0x7fc8240a4d50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc830009c60 tx=0x7fc830009260 comp rx=0 tx=0).stop 2024-03-24T03:00:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.461+0000 7fc841da6700 1 -- 172.21.15.17:0/3464685570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c1004d0 msgr2=0x7fc83c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.461+0000 7fc841da6700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c1004d0 0x7fc83c1983d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009f40 tx=0x7fc82c00c3a0 comp rx=0 tx=0).stop 2024-03-24T03:00:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.461+0000 7fc841da6700 1 -- 172.21.15.17:0/3464685570 shutdown_connections 2024-03-24T03:00:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.461+0000 7fc841da6700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8240a2890 0x7fc8240a4d50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.461+0000 7fc841da6700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c103e50 0x7fc83c19f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.461+0000 7fc841da6700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c1004d0 0x7fc83c1983d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.461+0000 7fc841da6700 1 --2- 172.21.15.17:0/3464685570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc83c1034b0 0x7fc83c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.461+0000 7fc841da6700 1 -- 172.21.15.17:0/3464685570 >> 172.21.15.17:0/3464685570 conn(0x7fc83c0fbe30 msgr2=0x7fc83c106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.461+0000 7fc841da6700 1 -- 172.21.15.17:0/3464685570 shutdown_connections 2024-03-24T03:00:26.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:26.461+0000 7fc841da6700 1 -- 172.21.15.17:0/3464685570 wait complete. 2024-03-24T03:00:27.155 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-24T03:00:27.404 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:00:28.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:27 smithi019 ceph-mon[35944]: pgmap v349: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:00:28.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:27 smithi019 ceph-mon[35944]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:28.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:27 smithi017 ceph-mon[31926]: pgmap v349: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:00:28.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:27 smithi017 ceph-mon[31926]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:28.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:27 smithi188 ceph-mon[36206]: pgmap v349: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:00:28.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:27 smithi188 ceph-mon[36206]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:29.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.261+0000 7f26eb972700 1 -- 172.21.15.17:0/3424700946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26e41021e0 msgr2=0x7f26e4102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:29.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.261+0000 7f26eb972700 1 --2- 172.21.15.17:0/3424700946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26e41021e0 0x7f26e4102600 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f26e00099c0 tx=0x7f26e0009cd0 comp rx=0 tx=0).stop 2024-03-24T03:00:29.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.261+0000 7f26eb972700 1 -- 172.21.15.17:0/3424700946 shutdown_connections 2024-03-24T03:00:29.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.261+0000 7f26eb972700 1 --2- 172.21.15.17:0/3424700946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e4103d80 0x7f26e410a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:29.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.261+0000 7f26eb972700 1 --2- 172.21.15.17:0/3424700946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26e41021e0 0x7f26e4102600 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:29.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.261+0000 7f26eb972700 1 --2- 172.21.15.17:0/3424700946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e41033e0 0x7f26e4103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.261+0000 7f26eb972700 1 -- 172.21.15.17:0/3424700946 >> 172.21.15.17:0/3424700946 conn(0x7f26e40fd7a0 msgr2=0x7f26e40ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.261+0000 7f26eb972700 1 -- 172.21.15.17:0/3424700946 shutdown_connections 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.261+0000 7f26eb972700 1 -- 172.21.15.17:0/3424700946 wait complete. 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26eb972700 1 Processor -- start 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26eb972700 1 -- start start 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26eb972700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e41021e0 0x7f26e41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26eb972700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e41033e0 0x7f26e41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26eb972700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26e4103d80 0x7f26e419f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26eb972700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e410ddd0 con 0x7f26e4103d80 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26eb972700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e410dc50 con 0x7f26e41021e0 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26eb972700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e410df50 con 0x7f26e41033e0 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26e9f0f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26e4103d80 0x7f26e419f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26e8f0d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e41033e0 0x7f26e41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:29.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26e970e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e41021e0 0x7f26e41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26e9f0f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26e4103d80 0x7f26e419f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35870/0 (socket says 172.21.15.17:35870) 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26e8f0d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e41033e0 0x7f26e41988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:57560/0 (socket says 172.21.15.17:57560) 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26e9f0f700 1 -- 172.21.15.17:0/408737346 learned_addr learned my addr 172.21.15.17:0/408737346 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26e970e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e41021e0 0x7f26e41983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:44826/0 (socket says 172.21.15.17:44826) 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26e9f0f700 1 -- 172.21.15.17:0/408737346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e41033e0 msgr2=0x7f26e41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26e9f0f700 1 --2- 172.21.15.17:0/408737346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e41033e0 0x7f26e41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26e9f0f700 1 -- 172.21.15.17:0/408737346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e41021e0 msgr2=0x7f26e41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26e9f0f700 1 --2- 172.21.15.17:0/408737346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e41021e0 0x7f26e41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.262+0000 7f26e9f0f700 1 -- 172.21.15.17:0/408737346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e41a0010 con 0x7f26e4103d80 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.263+0000 7f26e9f0f700 1 --2- 172.21.15.17:0/408737346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26e4103d80 0x7f26e419f970 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00ef60 tx=0x7f26dc00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.263+0000 7f26da7fc700 1 -- 172.21.15.17:0/408737346 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc0092e0 con 0x7f26e4103d80 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.263+0000 7f26da7fc700 1 -- 172.21.15.17:0/408737346 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26dc004bb0 con 0x7f26e4103d80 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.263+0000 7f26eb972700 1 -- 172.21.15.17:0/408737346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e41a02a0 con 0x7f26e4103d80 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.263+0000 7f26da7fc700 1 -- 172.21.15.17:0/408737346 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc004280 con 0x7f26e4103d80 2024-03-24T03:00:29.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.263+0000 7f26eb972700 1 -- 172.21.15.17:0/408737346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e41a07e0 con 0x7f26e4103d80 2024-03-24T03:00:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.264+0000 7f26eb972700 1 -- 172.21.15.17:0/408737346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e4068e50 con 0x7f26e4103d80 2024-03-24T03:00:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.265+0000 7f26da7fc700 1 -- 172.21.15.17:0/408737346 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f26dc01b030 con 0x7f26e4103d80 2024-03-24T03:00:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.265+0000 7f26da7fc700 1 --2- 172.21.15.17:0/408737346 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f26d00a2890 0x7f26d00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.266+0000 7f26e970e700 1 --2- 172.21.15.17:0/408737346 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f26d00a2890 0x7f26d00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.266+0000 7f26da7fc700 1 -- 172.21.15.17:0/408737346 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9050+0+0 (secure 0 0 0) 0x7f26dc0cf1b0 con 0x7f26e4103d80 2024-03-24T03:00:29.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.266+0000 7f26e970e700 1 --2- 172.21.15.17:0/408737346 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f26d00a2890 0x7f26d00a4d50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f26e000b580 tx=0x7f26e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:29.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.269+0000 7f26da7fc700 1 -- 172.21.15.17:0/408737346 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f26dc022020 con 0x7f26e4103d80 2024-03-24T03:00:29.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.483+0000 7f26eb972700 1 -- 172.21.15.17:0/408737346 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f26e40625d0 con 0x7f26d00a2890 2024-03-24T03:00:29.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.497+0000 7f26da7fc700 1 -- 172.21.15.17:0/408737346 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f26e40625d0 con 0x7f26d00a2890 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:mds.a smithi017 running (2m) 22s ago 2m 16.3M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 0b11fbe4e1a7 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:mds.b smithi019 running (118s) 52s ago 118s 19.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 0c7ee800196e 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:mds.c smithi188 running (2m) 52s ago 2m 20.1M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 aab8aa032050 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:mds.d smithi017 running (2m) 22s ago 2m 12.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 7a5f191e5486 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:mds.e smithi019 running (2m) 52s ago 2m 17.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2d477d3fb304 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:mds.f smithi188 running (2m) 52s ago 2m 20.1M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 8d20f8a515d9 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:mds.g smithi017 running (2m) 22s ago 2m 18.8M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e3ab97c6dd88 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:mds.h smithi019 running (2m) 52s ago 2m 19.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 35e3e6dbc206 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:mds.i smithi188 running (2m) 52s ago 2m 25.7M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 f3ce1e99fefa 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:mds.j smithi017 running (2m) 22s ago 2m 13.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 37509107601a 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:mds.k smithi019 running (2m) 52s ago 2m 16.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 535b6a63956f 2024-03-24T03:00:29.497 INFO:teuthology.orchestra.run.smithi017.stdout:mds.l smithi188 running (2m) 52s ago 2m 15.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 56139eb4601b 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:mgr.x smithi017 *:9283,8765 running (9m) 22s ago 9m 513M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 13a48e6ed072 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:mgr.y smithi019 *:8443,8765 running (7m) 52s ago 7m 452M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 283658810b5c 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:mgr.z smithi188 *:8443,8765 running (7m) 52s ago 7m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b80ef274913f 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:mon.a smithi017 running (9m) 22s ago 9m 52.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2644ce8a27f2 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:mon.b smithi019 running (8m) 52s ago 8m 42.9M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b65ede2e6d9 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:mon.c smithi188 running (8m) 52s ago 8m 40.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 6e9e3f618442 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0 smithi017 running (7m) 22s ago 7m 55.3M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 823ceb74ebc1 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1 smithi019 running (7m) 52s ago 7m 55.6M 4271M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9dd6c1770b03 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2 smithi188 running (6m) 52s ago 6m 55.0M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f106279c6228 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3 smithi017 running (6m) 22s ago 6m 55.0M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 7a991e11c046 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4 smithi019 running (5m) 52s ago 5m 50.9M 4271M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e59b6e74f5f0 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5 smithi188 running (5m) 52s ago 5m 52.6M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b2794db016f9 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:osd.6 smithi017 running (5m) 22s ago 5m 42.9M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9480c05fcace 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:osd.7 smithi019 running (4m) 52s ago 4m 44.6M 4271M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e47d8ce0b2a3 2024-03-24T03:00:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:osd.8 smithi188 running (4m) 52s ago 4m 52.1M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 068c762b871d 2024-03-24T03:00:29.499 INFO:teuthology.orchestra.run.smithi017.stdout:osd.9 smithi017 running (3m) 22s ago 3m 50.4M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 d27a961606df 2024-03-24T03:00:29.499 INFO:teuthology.orchestra.run.smithi017.stdout:osd.10 smithi019 running (3m) 52s ago 3m 43.8M 4271M 19.0.0-2145-g3be9a5c9 c664afe56bb0 7222f8a8ab5c 2024-03-24T03:00:29.499 INFO:teuthology.orchestra.run.smithi017.stdout:osd.11 smithi188 running (2m) 52s ago 2m 50.1M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 5543f5d3374b 2024-03-24T03:00:29.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.500+0000 7f26eb972700 1 -- 172.21.15.17:0/408737346 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f26d00a2890 msgr2=0x7f26d00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:29.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.500+0000 7f26eb972700 1 --2- 172.21.15.17:0/408737346 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f26d00a2890 0x7f26d00a4d50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f26e000b580 tx=0x7f26e0005fb0 comp rx=0 tx=0).stop 2024-03-24T03:00:29.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.500+0000 7f26eb972700 1 -- 172.21.15.17:0/408737346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26e4103d80 msgr2=0x7f26e419f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:29.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.500+0000 7f26eb972700 1 --2- 172.21.15.17:0/408737346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26e4103d80 0x7f26e419f970 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00ef60 tx=0x7f26dc00ef90 comp rx=0 tx=0).stop 2024-03-24T03:00:29.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.500+0000 7f26eb972700 1 -- 172.21.15.17:0/408737346 shutdown_connections 2024-03-24T03:00:29.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.500+0000 7f26eb972700 1 --2- 172.21.15.17:0/408737346 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f26d00a2890 0x7f26d00a4d50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:29.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.500+0000 7f26eb972700 1 --2- 172.21.15.17:0/408737346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26e4103d80 0x7f26e419f970 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:29.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.500+0000 7f26eb972700 1 --2- 172.21.15.17:0/408737346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e41021e0 0x7f26e41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:29.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.500+0000 7f26eb972700 1 --2- 172.21.15.17:0/408737346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e41033e0 0x7f26e41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:29.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.500+0000 7f26eb972700 1 -- 172.21.15.17:0/408737346 >> 172.21.15.17:0/408737346 conn(0x7f26e40fd7a0 msgr2=0x7f26e41066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:29.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.500+0000 7f26eb972700 1 -- 172.21.15.17:0/408737346 shutdown_connections 2024-03-24T03:00:29.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:29.500+0000 7f26eb972700 1 -- 172.21.15.17:0/408737346 wait complete. 2024-03-24T03:00:30.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:29 smithi019 ceph-mon[35944]: pgmap v350: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T03:00:30.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:29 smithi017 ceph-mon[31926]: pgmap v350: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T03:00:30.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:29 smithi188 ceph-mon[36206]: pgmap v350: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T03:00:30.334 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-24T03:00:30.586 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:00:30.959 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:30 smithi017 ceph-mon[31926]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:30.959 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-24T03:00:30.959 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 7, 6]}]: dispatch 2024-03-24T03:00:30.959 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-24T03:00:30.959 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]: dispatch 2024-03-24T03:00:30.959 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-24T03:00:30.959 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-03-24T03:00:30.960 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-03-24T03:00:30.960 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-03-24T03:00:30.960 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:30 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2024-03-24T03:00:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:30 smithi019 ceph-mon[35944]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-24T03:00:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 7, 6]}]: dispatch 2024-03-24T03:00:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-24T03:00:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]: dispatch 2024-03-24T03:00:31.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-24T03:00:31.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-03-24T03:00:31.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-03-24T03:00:31.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-03-24T03:00:31.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:30 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2024-03-24T03:00:31.129 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:30 smithi188 ceph-mon[36206]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:31.129 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-24T03:00:31.130 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 7, 6]}]: dispatch 2024-03-24T03:00:31.130 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-24T03:00:31.130 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]: dispatch 2024-03-24T03:00:31.130 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-24T03:00:31.130 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-03-24T03:00:31.130 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-03-24T03:00:31.130 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-03-24T03:00:31.130 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:30 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2024-03-24T03:00:32.080 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:31 smithi017 ceph-mon[31926]: pgmap v351: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-24T03:00:32.080 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-24T03:00:32.080 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 7, 6]}]': finished 2024-03-24T03:00:32.080 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-24T03:00:32.080 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]': finished 2024-03-24T03:00:32.080 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-24T03:00:32.080 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-03-24T03:00:32.080 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-03-24T03:00:32.080 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-03-24T03:00:32.080 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:31 smithi017 ceph-mon[31926]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2024-03-24T03:00:32.080 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:31 smithi017 ceph-mon[31926]: osdmap e92: 12 total, 12 up, 12 in 2024-03-24T03:00:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:31 smithi188 ceph-mon[36206]: pgmap v351: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-24T03:00:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-24T03:00:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 7, 6]}]': finished 2024-03-24T03:00:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-24T03:00:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]': finished 2024-03-24T03:00:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-24T03:00:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-03-24T03:00:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-03-24T03:00:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-03-24T03:00:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:31 smithi188 ceph-mon[36206]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2024-03-24T03:00:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:31 smithi188 ceph-mon[36206]: osdmap e92: 12 total, 12 up, 12 in 2024-03-24T03:00:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:31 smithi019 ceph-mon[35944]: pgmap v351: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-24T03:00:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-24T03:00:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 7, 6]}]': finished 2024-03-24T03:00:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-24T03:00:32.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]': finished 2024-03-24T03:00:32.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-24T03:00:32.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-03-24T03:00:32.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-03-24T03:00:32.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-03-24T03:00:32.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:31 smithi019 ceph-mon[35944]: from='mgr.14152 172.21.15.17:0/759801864' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2024-03-24T03:00:32.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:31 smithi019 ceph-mon[35944]: osdmap e92: 12 total, 12 up, 12 in 2024-03-24T03:00:32.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.423+0000 7f8d6739f700 1 -- 172.21.15.17:0/4018826788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601032e0 msgr2=0x7f8d60103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:32.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.423+0000 7f8d6739f700 1 --2- 172.21.15.17:0/4018826788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601032e0 0x7f8d60103740 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8d540099c0 tx=0x7f8d54009cd0 comp rx=0 tx=0).stop 2024-03-24T03:00:32.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.423+0000 7f8d6739f700 1 -- 172.21.15.17:0/4018826788 shutdown_connections 2024-03-24T03:00:32.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.423+0000 7f8d6739f700 1 --2- 172.21.15.17:0/4018826788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d60103c80 0x7f8d6010a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:32.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.423+0000 7f8d6739f700 1 --2- 172.21.15.17:0/4018826788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d601020e0 0x7f8d60102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:32.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.423+0000 7f8d6739f700 1 --2- 172.21.15.17:0/4018826788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601032e0 0x7f8d60103740 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:32.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.423+0000 7f8d6739f700 1 -- 172.21.15.17:0/4018826788 >> 172.21.15.17:0/4018826788 conn(0x7f8d600fd6a0 msgr2=0x7f8d600ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:32.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.423+0000 7f8d6739f700 1 -- 172.21.15.17:0/4018826788 shutdown_connections 2024-03-24T03:00:32.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.423+0000 7f8d6739f700 1 -- 172.21.15.17:0/4018826788 wait complete. 2024-03-24T03:00:32.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d6739f700 1 Processor -- start 2024-03-24T03:00:32.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d6739f700 1 -- start start 2024-03-24T03:00:32.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d6739f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601020e0 0x7f8d60198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:32.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d6739f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d601032e0 0x7f8d601987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:32.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d6739f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d60103c80 0x7f8d6019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d6739f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6010dcc0 con 0x7f8d601020e0 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d6739f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6010db40 con 0x7f8d60103c80 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d6739f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6010de40 con 0x7f8d601032e0 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d6639d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601020e0 0x7f8d60198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d6639d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601020e0 0x7f8d60198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41268/0 (socket says 172.21.15.17:41268) 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d6639d700 1 -- 172.21.15.17:0/361393060 learned_addr learned my addr 172.21.15.17:0/361393060 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d65b9c700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d601032e0 0x7f8d601987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.424+0000 7f8d66b9e700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d60103c80 0x7f8d6019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.425+0000 7f8d6639d700 1 -- 172.21.15.17:0/361393060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d601032e0 msgr2=0x7f8d601987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.425+0000 7f8d6639d700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d601032e0 0x7f8d601987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.425+0000 7f8d6639d700 1 -- 172.21.15.17:0/361393060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d60103c80 msgr2=0x7f8d6019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.425+0000 7f8d6639d700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d60103c80 0x7f8d6019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:32.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.425+0000 7f8d6639d700 1 -- 172.21.15.17:0/361393060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6019fed0 con 0x7f8d601020e0 2024-03-24T03:00:32.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.425+0000 7f8d6639d700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601020e0 0x7f8d60198270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00e980 tx=0x7f8d5c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:32.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.425+0000 7f8d537fe700 1 -- 172.21.15.17:0/361393060 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c004230 con 0x7f8d601020e0 2024-03-24T03:00:32.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.425+0000 7f8d537fe700 1 -- 172.21.15.17:0/361393060 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d5c0043d0 con 0x7f8d601020e0 2024-03-24T03:00:32.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.425+0000 7f8d6739f700 1 -- 172.21.15.17:0/361393060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d601a01c0 con 0x7f8d601020e0 2024-03-24T03:00:32.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.425+0000 7f8d6739f700 1 -- 172.21.15.17:0/361393060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d601a06b0 con 0x7f8d601020e0 2024-03-24T03:00:32.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.425+0000 7f8d537fe700 1 -- 172.21.15.17:0/361393060 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c003dc0 con 0x7f8d601020e0 2024-03-24T03:00:32.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.426+0000 7f8d6739f700 1 -- 172.21.15.17:0/361393060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d60068e50 con 0x7f8d601020e0 2024-03-24T03:00:32.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.427+0000 7f8d537fe700 1 -- 172.21.15.17:0/361393060 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8d5c00c810 con 0x7f8d601020e0 2024-03-24T03:00:32.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.428+0000 7f8d537fe700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8d4c0a2890 0x7f8d4c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:32.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.428+0000 7f8d65b9c700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8d4c0a2890 0x7f8d4c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:32.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.428+0000 7f8d537fe700 1 -- 172.21.15.17:0/361393060 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f8d5c0cf100 con 0x7f8d601020e0 2024-03-24T03:00:32.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.429+0000 7f8d65b9c700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8d4c0a2890 0x7f8d4c0a4d50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d54000c00 tx=0x7f8d54005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:32.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.433+0000 7f8d537fe700 1 -- 172.21.15.17:0/361393060 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8d5c09bd60 con 0x7f8d601020e0 2024-03-24T03:00:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.649+0000 7f8d6739f700 1 -- 172.21.15.17:0/361393060 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d600625d0 con 0x7f8d4c0a2890 2024-03-24T03:00:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.655+0000 7f8d537fe700 1 -- 172.21.15.17:0/361393060 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f8d600625d0 con 0x7f8d4c0a2890 2024-03-24T03:00:32.655 INFO:teuthology.orchestra.run.smithi017.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-24T03:00:32.655 INFO:teuthology.orchestra.run.smithi017.stdout:mds.all 12/12 55s ago 2m smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi019=b;smithi019=e;smithi019=h;smithi019=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-03-24T03:00:32.655 INFO:teuthology.orchestra.run.smithi017.stdout:mgr 3/3 55s ago 7m smithi017=x;smithi019=y;smithi188=z;count:3 2024-03-24T03:00:32.655 INFO:teuthology.orchestra.run.smithi017.stdout:mon 3/3 55s ago 8m smithi017:172.21.15.17=a;smithi019:172.21.15.19=b;smithi188:172.21.15.188=c;count:3 2024-03-24T03:00:32.655 INFO:teuthology.orchestra.run.smithi017.stdout:osd 12 55s ago - 2024-03-24T03:00:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.657+0000 7f8d6739f700 1 -- 172.21.15.17:0/361393060 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8d4c0a2890 msgr2=0x7f8d4c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.657+0000 7f8d6739f700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8d4c0a2890 0x7f8d4c0a4d50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d54000c00 tx=0x7f8d54005fb0 comp rx=0 tx=0).stop 2024-03-24T03:00:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.657+0000 7f8d6739f700 1 -- 172.21.15.17:0/361393060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601020e0 msgr2=0x7f8d60198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.657+0000 7f8d6739f700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601020e0 0x7f8d60198270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00e980 tx=0x7f8d5c00ec90 comp rx=0 tx=0).stop 2024-03-24T03:00:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.658+0000 7f8d6739f700 1 -- 172.21.15.17:0/361393060 shutdown_connections 2024-03-24T03:00:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.658+0000 7f8d6739f700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f8d4c0a2890 0x7f8d4c0a4d50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.658+0000 7f8d6739f700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d60103c80 0x7f8d6019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.658+0000 7f8d6739f700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d601020e0 0x7f8d60198270 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.658+0000 7f8d6739f700 1 --2- 172.21.15.17:0/361393060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d601032e0 0x7f8d601987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.658+0000 7f8d6739f700 1 -- 172.21.15.17:0/361393060 >> 172.21.15.17:0/361393060 conn(0x7f8d600fd6a0 msgr2=0x7f8d601065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.658+0000 7f8d6739f700 1 -- 172.21.15.17:0/361393060 shutdown_connections 2024-03-24T03:00:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:32.658+0000 7f8d6739f700 1 -- 172.21.15.17:0/361393060 wait complete. 2024-03-24T03:00:32.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:32 smithi017 ceph-mon[31926]: osdmap e93: 12 total, 12 up, 12 in 2024-03-24T03:00:33.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:32 smithi188 ceph-mon[36206]: osdmap e93: 12 total, 12 up, 12 in 2024-03-24T03:00:33.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:32 smithi019 ceph-mon[35944]: osdmap e93: 12 total, 12 up, 12 in 2024-03-24T03:00:33.457 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-24T03:00:33.704 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:00:34.079 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:33 smithi017 ceph-mon[31926]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s 2024-03-24T03:00:34.079 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:33 smithi017 ceph-mon[31926]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:34.079 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:33 smithi017 ceph-mon[31926]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-24T03:00:34.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:33 smithi188 ceph-mon[36206]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s 2024-03-24T03:00:34.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:33 smithi188 ceph-mon[36206]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:33 smithi188 ceph-mon[36206]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-24T03:00:34.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:33 smithi019 ceph-mon[35944]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s 2024-03-24T03:00:34.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:33 smithi019 ceph-mon[35944]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:34.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:33 smithi019 ceph-mon[35944]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-24T03:00:35.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.545+0000 7f71935e2700 1 -- 172.21.15.17:0/2016669751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c1034b0 msgr2=0x7f718c103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:35.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.545+0000 7f71935e2700 1 --2- 172.21.15.17:0/2016669751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c1034b0 0x7f718c103910 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7180009990 tx=0x7f7180009ca0 comp rx=0 tx=0).stop 2024-03-24T03:00:35.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.545+0000 7f71935e2700 1 -- 172.21.15.17:0/2016669751 shutdown_connections 2024-03-24T03:00:35.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.545+0000 7f71935e2700 1 --2- 172.21.15.17:0/2016669751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c103e50 0x7f718c10a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:35.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.545+0000 7f71935e2700 1 --2- 172.21.15.17:0/2016669751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f718c1004d0 0x7f718c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:35.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.545+0000 7f71935e2700 1 --2- 172.21.15.17:0/2016669751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c1034b0 0x7f718c103910 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:35.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.545+0000 7f71935e2700 1 -- 172.21.15.17:0/2016669751 >> 172.21.15.17:0/2016669751 conn(0x7f718c0fbe30 msgr2=0x7f718c0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.545+0000 7f71935e2700 1 -- 172.21.15.17:0/2016669751 shutdown_connections 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.545+0000 7f71935e2700 1 -- 172.21.15.17:0/2016669751 wait complete. 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f71935e2700 1 Processor -- start 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f71935e2700 1 -- start start 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f71935e2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f718c1004d0 0x7f718c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f71935e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c1034b0 0x7f718c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f71935e2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c103e50 0x7f718c19f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f71935e2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c10dde0 con 0x7f718c103e50 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f71935e2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c10dc60 con 0x7f718c1034b0 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f71935e2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c10df60 con 0x7f718c1004d0 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f719137e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f718c1004d0 0x7f718c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f719137e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f718c1004d0 0x7f718c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:50754/0 (socket says 172.21.15.17:50754) 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f7190b7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c1034b0 0x7f718c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:35.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f719137e700 1 -- 172.21.15.17:0/1403753994 learned_addr learned my addr 172.21.15.17:0/1403753994 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:35.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f7191b7f700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c103e50 0x7f718c19f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:35.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f719137e700 1 -- 172.21.15.17:0/1403753994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c1034b0 msgr2=0x7f718c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:35.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f719137e700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c1034b0 0x7f718c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:35.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f719137e700 1 -- 172.21.15.17:0/1403753994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c103e50 msgr2=0x7f718c19f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:35.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f719137e700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c103e50 0x7f718c19f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:35.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f719137e700 1 -- 172.21.15.17:0/1403753994 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c1a0030 con 0x7f718c1004d0 2024-03-24T03:00:35.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.546+0000 7f7190b7d700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c1034b0 0x7f718c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:35.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.547+0000 7f719137e700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f718c1004d0 0x7f718c1983d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7184009f20 tx=0x7f718400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:35.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.547+0000 7f718a7fc700 1 -- 172.21.15.17:0/1403753994 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184003a00 con 0x7f718c1004d0 2024-03-24T03:00:35.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.547+0000 7f71935e2700 1 -- 172.21.15.17:0/1403753994 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718c1a0320 con 0x7f718c1004d0 2024-03-24T03:00:35.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.547+0000 7f718a7fc700 1 -- 172.21.15.17:0/1403753994 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7184003ba0 con 0x7f718c1004d0 2024-03-24T03:00:35.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.547+0000 7f718a7fc700 1 -- 172.21.15.17:0/1403753994 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184004890 con 0x7f718c1004d0 2024-03-24T03:00:35.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.547+0000 7f71935e2700 1 -- 172.21.15.17:0/1403753994 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718c1a0810 con 0x7f718c1004d0 2024-03-24T03:00:35.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.548+0000 7f71935e2700 1 -- 172.21.15.17:0/1403753994 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718c068e50 con 0x7f718c1004d0 2024-03-24T03:00:35.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.549+0000 7f718a7fc700 1 -- 172.21.15.17:0/1403753994 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7184012050 con 0x7f718c1004d0 2024-03-24T03:00:35.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.550+0000 7f718a7fc700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f71740a2890 0x7f71740a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:35.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.550+0000 7f7190b7d700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f71740a2890 0x7f71740a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:35.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.550+0000 7f718a7fc700 1 -- 172.21.15.17:0/1403753994 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f71840cec80 con 0x7f718c1004d0 2024-03-24T03:00:35.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.550+0000 7f7190b7d700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f71740a2890 0x7f71740a4d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f718000b2e0 tx=0x7f718000b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:35.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.559+0000 7f718a7fc700 1 -- 172.21.15.17:0/1403753994 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7184010040 con 0x7f718c1004d0 2024-03-24T03:00:35.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.771+0000 7f71935e2700 1 -- 172.21.15.17:0/1403753994 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f718c062690 con 0x7f71740a2890 2024-03-24T03:00:35.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.773+0000 7f718a7fc700 1 -- 172.21.15.17:0/1403753994 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f718c062690 con 0x7f71740a2890 2024-03-24T03:00:35.773 INFO:teuthology.orchestra.run.smithi017.stdout:HOST ADDR LABELS STATUS 2024-03-24T03:00:35.773 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017 172.21.15.17 2024-03-24T03:00:35.773 INFO:teuthology.orchestra.run.smithi017.stdout:smithi019 172.21.15.19 2024-03-24T03:00:35.773 INFO:teuthology.orchestra.run.smithi017.stdout:smithi188 172.21.15.188 2024-03-24T03:00:35.773 INFO:teuthology.orchestra.run.smithi017.stdout:3 hosts in cluster 2024-03-24T03:00:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.775+0000 7f71935e2700 1 -- 172.21.15.17:0/1403753994 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f71740a2890 msgr2=0x7f71740a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.775+0000 7f71935e2700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f71740a2890 0x7f71740a4d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f718000b2e0 tx=0x7f718000b880 comp rx=0 tx=0).stop 2024-03-24T03:00:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.775+0000 7f71935e2700 1 -- 172.21.15.17:0/1403753994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f718c1004d0 msgr2=0x7f718c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.775+0000 7f71935e2700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f718c1004d0 0x7f718c1983d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7184009f20 tx=0x7f718400c3a0 comp rx=0 tx=0).stop 2024-03-24T03:00:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.775+0000 7f71935e2700 1 -- 172.21.15.17:0/1403753994 shutdown_connections 2024-03-24T03:00:35.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.775+0000 7f71935e2700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f71740a2890 0x7f71740a4d50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:35.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.775+0000 7f71935e2700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c103e50 0x7f718c19f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:35.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.775+0000 7f71935e2700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f718c1004d0 0x7f718c1983d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:35.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.775+0000 7f71935e2700 1 --2- 172.21.15.17:0/1403753994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c1034b0 0x7f718c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:35.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.775+0000 7f71935e2700 1 -- 172.21.15.17:0/1403753994 >> 172.21.15.17:0/1403753994 conn(0x7f718c0fbe30 msgr2=0x7f718c106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:35.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.775+0000 7f71935e2700 1 -- 172.21.15.17:0/1403753994 shutdown_connections 2024-03-24T03:00:35.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:35.775+0000 7f71935e2700 1 -- 172.21.15.17:0/1403753994 wait complete. 2024-03-24T03:00:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:35 smithi017 ceph-mon[31926]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s; 8.2 KiB/s, 1 objects/s recovering 2024-03-24T03:00:36.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:35 smithi188 ceph-mon[36206]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s; 8.2 KiB/s, 1 objects/s recovering 2024-03-24T03:00:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:35 smithi019 ceph-mon[35944]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s; 8.2 KiB/s, 1 objects/s recovering 2024-03-24T03:00:36.647 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-24T03:00:36.897 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:00:36.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:36 smithi017 ceph-mon[31926]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:37.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:36 smithi188 ceph-mon[36206]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:37.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:36 smithi019 ceph-mon[35944]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:38.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:37 smithi017 ceph-mon[31926]: pgmap v356: 129 pgs: 2 peering, 127 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 7 op/s; 8.2 KiB/s, 1 objects/s recovering 2024-03-24T03:00:38.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:37 smithi188 ceph-mon[36206]: pgmap v356: 129 pgs: 2 peering, 127 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 7 op/s; 8.2 KiB/s, 1 objects/s recovering 2024-03-24T03:00:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:37 smithi019 ceph-mon[35944]: pgmap v356: 129 pgs: 2 peering, 127 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 7 op/s; 8.2 KiB/s, 1 objects/s recovering 2024-03-24T03:00:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.578+0000 7f2ae9e70700 1 -- 172.21.15.17:0/519715852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ae41033c0 msgr2=0x7f2ae4103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.578+0000 7f2ae9e70700 1 --2- 172.21.15.17:0/519715852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ae41033c0 0x7f2ae4103820 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80099c0 tx=0x7f2ad8009cd0 comp rx=0 tx=0).stop 2024-03-24T03:00:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.578+0000 7f2ae9e70700 1 -- 172.21.15.17:0/519715852 shutdown_connections 2024-03-24T03:00:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.578+0000 7f2ae9e70700 1 --2- 172.21.15.17:0/519715852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ae4103d60 0x7f2ae410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.578+0000 7f2ae9e70700 1 --2- 172.21.15.17:0/519715852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ae41021c0 0x7f2ae41025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.578+0000 7f2ae9e70700 1 --2- 172.21.15.17:0/519715852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ae41033c0 0x7f2ae4103820 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.578+0000 7f2ae9e70700 1 -- 172.21.15.17:0/519715852 >> 172.21.15.17:0/519715852 conn(0x7f2ae40fd7a0 msgr2=0x7f2ae40ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.578+0000 7f2ae9e70700 1 -- 172.21.15.17:0/519715852 shutdown_connections 2024-03-24T03:00:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.578+0000 7f2ae9e70700 1 -- 172.21.15.17:0/519715852 wait complete. 2024-03-24T03:00:38.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae9e70700 1 Processor -- start 2024-03-24T03:00:38.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae9e70700 1 -- start start 2024-03-24T03:00:38.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae9e70700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ae41021c0 0x7f2ae4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:38.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae9e70700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ae41033c0 0x7f2ae4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:38.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae9e70700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ae4103d60 0x7f2ae419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:38.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae9e70700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae410ddd0 con 0x7f2ae41021c0 2024-03-24T03:00:38.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae9e70700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae410dc50 con 0x7f2ae4103d60 2024-03-24T03:00:38.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae9e70700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae410df50 con 0x7f2ae41033c0 2024-03-24T03:00:38.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ae41021c0 0x7f2ae4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:38.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ae41021c0 0x7f2ae4198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41308/0 (socket says 172.21.15.17:41308) 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae37fe700 1 -- 172.21.15.17:0/538181909 learned_addr learned my addr 172.21.15.17:0/538181909 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae2ffd700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ae41033c0 0x7f2ae4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.579+0000 7f2ae3fff700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ae4103d60 0x7f2ae419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.580+0000 7f2ae37fe700 1 -- 172.21.15.17:0/538181909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ae41033c0 msgr2=0x7f2ae4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.580+0000 7f2ae37fe700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ae41033c0 0x7f2ae4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.580+0000 7f2ae37fe700 1 -- 172.21.15.17:0/538181909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ae4103d60 msgr2=0x7f2ae419f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.580+0000 7f2ae37fe700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ae4103d60 0x7f2ae419f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.580+0000 7f2ae37fe700 1 -- 172.21.15.17:0/538181909 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae41a0070 con 0x7f2ae41021c0 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.580+0000 7f2ae37fe700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ae41021c0 0x7f2ae4198410 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400e9a0 tx=0x7f2ad400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.580+0000 7f2ae0ff9700 1 -- 172.21.15.17:0/538181909 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4004220 con 0x7f2ae41021c0 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.580+0000 7f2ae9e70700 1 -- 172.21.15.17:0/538181909 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae41a0360 con 0x7f2ae41021c0 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.580+0000 7f2ae0ff9700 1 -- 172.21.15.17:0/538181909 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ad40043c0 con 0x7f2ae41021c0 2024-03-24T03:00:38.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.580+0000 7f2ae0ff9700 1 -- 172.21.15.17:0/538181909 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4003df0 con 0x7f2ae41021c0 2024-03-24T03:00:38.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.580+0000 7f2ae9e70700 1 -- 172.21.15.17:0/538181909 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae41a08f0 con 0x7f2ae41021c0 2024-03-24T03:00:38.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.581+0000 7f2ae9e70700 1 -- 172.21.15.17:0/538181909 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae4068e50 con 0x7f2ae41021c0 2024-03-24T03:00:38.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.582+0000 7f2ae0ff9700 1 -- 172.21.15.17:0/538181909 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2ad400c810 con 0x7f2ae41021c0 2024-03-24T03:00:38.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.583+0000 7f2ae0ff9700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2acc0a2890 0x7f2acc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:38.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.583+0000 7f2ae2ffd700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2acc0a2890 0x7f2acc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:38.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.583+0000 7f2ae0ff9700 1 -- 172.21.15.17:0/538181909 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f2ad40cf170 con 0x7f2ae41021c0 2024-03-24T03:00:38.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.583+0000 7f2ae2ffd700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2acc0a2890 0x7f2acc0a4d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8000c00 tx=0x7f2ad8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:38.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.586+0000 7f2ae0ff9700 1 -- 172.21.15.17:0/538181909 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2ad409be60 con 0x7f2ae41021c0 2024-03-24T03:00:38.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.799+0000 7f2ae9e70700 1 -- 172.21.15.17:0/538181909 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ae40625d0 con 0x7f2acc0a2890 2024-03-24T03:00:38.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.800+0000 7f2ae0ff9700 1 -- 172.21.15.17:0/538181909 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f2ae40625d0 con 0x7f2acc0a2890 2024-03-24T03:00:38.801 INFO:teuthology.orchestra.run.smithi017.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-24T03:00:38.801 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7473003Q480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-24T03:00:38.801 INFO:teuthology.orchestra.run.smithi017.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-24T03:00:38.801 INFO:teuthology.orchestra.run.smithi017.stdout:smithi188 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MP400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-24T03:00:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.802+0000 7f2ae9e70700 1 -- 172.21.15.17:0/538181909 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2acc0a2890 msgr2=0x7f2acc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.803+0000 7f2ae9e70700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2acc0a2890 0x7f2acc0a4d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8000c00 tx=0x7f2ad8005fb0 comp rx=0 tx=0).stop 2024-03-24T03:00:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.803+0000 7f2ae9e70700 1 -- 172.21.15.17:0/538181909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ae41021c0 msgr2=0x7f2ae4198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.803+0000 7f2ae9e70700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ae41021c0 0x7f2ae4198410 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400e9a0 tx=0x7f2ad400ecb0 comp rx=0 tx=0).stop 2024-03-24T03:00:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.803+0000 7f2ae9e70700 1 -- 172.21.15.17:0/538181909 shutdown_connections 2024-03-24T03:00:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.803+0000 7f2ae9e70700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f2acc0a2890 0x7f2acc0a4d50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.803+0000 7f2ae9e70700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ae4103d60 0x7f2ae419f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:38.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.803+0000 7f2ae9e70700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ae41021c0 0x7f2ae4198410 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.803+0000 7f2ae9e70700 1 --2- 172.21.15.17:0/538181909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ae41033c0 0x7f2ae4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.803+0000 7f2ae9e70700 1 -- 172.21.15.17:0/538181909 >> 172.21.15.17:0/538181909 conn(0x7f2ae40fd7a0 msgr2=0x7f2ae4106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.803+0000 7f2ae9e70700 1 -- 172.21.15.17:0/538181909 shutdown_connections 2024-03-24T03:00:38.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:38.803+0000 7f2ae9e70700 1 -- 172.21.15.17:0/538181909 wait complete. 2024-03-24T03:00:39.191 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:38 smithi017 ceph-mon[31926]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-24T03:00:39.191 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:38 smithi017 ceph-mon[31926]: Cluster is now healthy 2024-03-24T03:00:39.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:38 smithi188 ceph-mon[36206]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-24T03:00:39.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:38 smithi188 ceph-mon[36206]: Cluster is now healthy 2024-03-24T03:00:39.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:38 smithi019 ceph-mon[35944]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-24T03:00:39.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:38 smithi019 ceph-mon[35944]: Cluster is now healthy 2024-03-24T03:00:39.504 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T03:00:39.512 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2024-03-24T03:00:39.513 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-24T03:00:39.761 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:00:40.204 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:39 smithi017 ceph-mon[31926]: pgmap v357: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s; 8.3 KiB/s, 2 objects/s recovering 2024-03-24T03:00:40.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:39 smithi017 ceph-mon[31926]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:40.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:39 smithi188 ceph-mon[36206]: pgmap v357: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s; 8.3 KiB/s, 2 objects/s recovering 2024-03-24T03:00:40.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:39 smithi188 ceph-mon[36206]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:39 smithi019 ceph-mon[35944]: pgmap v357: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s; 8.3 KiB/s, 2 objects/s recovering 2024-03-24T03:00:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:39 smithi019 ceph-mon[35944]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:00:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.456+0000 7f7870d9e700 1 -- 172.21.15.17:0/986920175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c103c80 msgr2=0x7f786c10a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.456+0000 7f7870d9e700 1 --2- 172.21.15.17:0/986920175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c103c80 0x7f786c10a530 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f786400b0d0 tx=0x7f786400b3e0 comp rx=0 tx=0).stop 2024-03-24T03:00:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.456+0000 7f7870d9e700 1 -- 172.21.15.17:0/986920175 shutdown_connections 2024-03-24T03:00:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.456+0000 7f7870d9e700 1 --2- 172.21.15.17:0/986920175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c103c80 0x7f786c10a530 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.456+0000 7f7870d9e700 1 --2- 172.21.15.17:0/986920175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f786c1020e0 0x7f786c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.456+0000 7f7870d9e700 1 --2- 172.21.15.17:0/986920175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f786c1032e0 0x7f786c103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.456+0000 7f7870d9e700 1 -- 172.21.15.17:0/986920175 >> 172.21.15.17:0/986920175 conn(0x7f786c0fd6a0 msgr2=0x7f786c0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.456+0000 7f7870d9e700 1 -- 172.21.15.17:0/986920175 shutdown_connections 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.456+0000 7f7870d9e700 1 -- 172.21.15.17:0/986920175 wait complete. 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.457+0000 7f7870d9e700 1 Processor -- start 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.457+0000 7f7870d9e700 1 -- start start 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f7870d9e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f786c1020e0 0x7f786c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f7870d9e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f786c1032e0 0x7f786c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f7870d9e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c103c80 0x7f786c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f7870d9e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c10dcb0 con 0x7f786c103c80 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f7870d9e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c10db30 con 0x7f786c1032e0 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f7870d9e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c10de30 con 0x7f786c1020e0 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f786bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c103c80 0x7f786c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f786b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f786c1020e0 0x7f786c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f786bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c103c80 0x7f786c19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41568/0 (socket says 172.21.15.17:41568) 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f786b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f786c1020e0 0x7f786c198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:42612/0 (socket says 172.21.15.17:42612) 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f786bfff700 1 -- 172.21.15.17:0/3269364169 learned_addr learned my addr 172.21.15.17:0/3269364169 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f7863fff700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f786c1032e0 0x7f786c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f786b7fe700 1 -- 172.21.15.17:0/3269364169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f786c1032e0 msgr2=0x7f786c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f786b7fe700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f786c1032e0 0x7f786c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f786b7fe700 1 -- 172.21.15.17:0/3269364169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c103c80 msgr2=0x7f786c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f786b7fe700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c103c80 0x7f786c19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f786b7fe700 1 -- 172.21.15.17:0/3269364169 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c19fe90 con 0x7f786c1020e0 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f786bfff700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c103c80 0x7f786c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f7863fff700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f786c1032e0 0x7f786c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f786b7fe700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f786c1020e0 0x7f786c198230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f785c00e9f0 tx=0x7f785c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.458+0000 7f78697fa700 1 -- 172.21.15.17:0/3269364169 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c004220 con 0x7f786c1020e0 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.459+0000 7f7870d9e700 1 -- 172.21.15.17:0/3269364169 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c1a0180 con 0x7f786c1020e0 2024-03-24T03:00:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.459+0000 7f78697fa700 1 -- 172.21.15.17:0/3269364169 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f785c0043c0 con 0x7f786c1020e0 2024-03-24T03:00:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.459+0000 7f78697fa700 1 -- 172.21.15.17:0/3269364169 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c003cb0 con 0x7f786c1020e0 2024-03-24T03:00:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.459+0000 7f7870d9e700 1 -- 172.21.15.17:0/3269364169 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c1a0710 con 0x7f786c1020e0 2024-03-24T03:00:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.459+0000 7f7870d9e700 1 -- 172.21.15.17:0/3269364169 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c068e50 con 0x7f786c1020e0 2024-03-24T03:00:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.461+0000 7f78697fa700 1 -- 172.21.15.17:0/3269364169 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f785c00c7c0 con 0x7f786c1020e0 2024-03-24T03:00:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.462+0000 7f78697fa700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f78580a2890 0x7f78580a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.462+0000 7f7863fff700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f78580a2890 0x7f78580a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.462+0000 7f78697fa700 1 -- 172.21.15.17:0/3269364169 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f785c0cf170 con 0x7f786c1020e0 2024-03-24T03:00:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.462+0000 7f7863fff700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f78580a2890 0x7f78580a4d50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7854011490 tx=0x7f7854009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:41.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.469+0000 7f78697fa700 1 -- 172.21.15.17:0/3269364169 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f785c09be00 con 0x7f786c1020e0 2024-03-24T03:00:41.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.720+0000 7f7870d9e700 1 -- 172.21.15.17:0/3269364169 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f786c062da0 con 0x7f786c1020e0 2024-03-24T03:00:41.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.720+0000 7f78697fa700 1 -- 172.21.15.17:0/3269364169 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+3033 (secure 0 0 0) 0x7f785c09b700 con 0x7f786c1020e0 2024-03-24T03:00:41.721 INFO:teuthology.orchestra.run.smithi017.stdout:e28 2024-03-24T03:00:41.721 INFO:teuthology.orchestra.run.smithi017.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T03:00:41.721 INFO:teuthology.orchestra.run.smithi017.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-24T03:00:41.721 INFO:teuthology.orchestra.run.smithi017.stdout:legacy client fscid: 1 2024-03-24T03:00:41.721 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:41.721 INFO:teuthology.orchestra.run.smithi017.stdout:Filesystem 'cephfs' (1) 2024-03-24T03:00:41.721 INFO:teuthology.orchestra.run.smithi017.stdout:fs_name cephfs 2024-03-24T03:00:41.721 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 28 2024-03-24T03:00:41.721 INFO:teuthology.orchestra.run.smithi017.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:created 2024-03-24T02:58:36.476051+0000 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:modified 2024-03-24T02:58:59.074424+0000 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:tableserver 0 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:root 0 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:session_timeout 300 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:session_autoclose 300 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:max_file_size 1099511627776 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:max_xattr_size 65536 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:required_client_features {} 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:last_failure 0 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:last_failure_osd_epoch 0 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-24T03:00:41.722 INFO:teuthology.orchestra.run.smithi017.stdout:max_mds 5 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:in 0,1,2,3,4 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:up {0=24529,1=24521,2=24511,3=24493,4=24485} 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:failed 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:damaged 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:stopped 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:data_pools [3,4] 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:metadata_pool 2 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:inline_data disabled 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:balancer 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:bal_rank_mask -1 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:standby_count_wanted 0 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:qdb_cluster leader: 24529 members: 24511,24529,24485,24493,24521 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.i{0:24529} state up:active seq 5 addr [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T03:00:41.723 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.e{0:24452} state up:standby-replay seq 1 addr [v2:172.21.15.19:6834/1025006466,v1:172.21.15.19:6835/1025006466] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b{1:24521} state up:active seq 8 addr [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.l{1:24448} state up:standby-replay seq 1 addr [v2:172.21.15.188:6832/920982807,v1:172.21.15.188:6833/920982807] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.c{2:24511} state up:active seq 11 addr [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.k{2:24419} state up:standby-replay seq 1 addr [v2:172.21.15.19:6832/3469034636,v1:172.21.15.19:6833/3469034636] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.f{3:24493} state up:active seq 12 addr [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a{3:14643} state up:standby-replay seq 1 addr [v2:172.21.15.17:6840/1592961709,v1:172.21.15.17:6841/1592961709] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.h{4:24485} state up:active seq 11 addr [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.g{4:14631} state up:standby-replay seq 1 addr [v2:172.21.15.17:6838/2895019504,v1:172.21.15.17:6839/2895019504] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout:Standby daemons: 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:41.724 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.j{-1:14556} state up:standby seq 1 addr [v2:172.21.15.17:6834/2854025639,v1:172.21.15.17:6835/2854025639] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T03:00:41.725 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.d{-1:14568} state up:standby seq 1 addr [v2:172.21.15.17:6836/3530669946,v1:172.21.15.17:6837/3530669946] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T03:00:41.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.723+0000 7f7870d9e700 1 -- 172.21.15.17:0/3269364169 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f78580a2890 msgr2=0x7f78580a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:41.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.723+0000 7f7870d9e700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f78580a2890 0x7f78580a4d50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7854011490 tx=0x7f7854009260 comp rx=0 tx=0).stop 2024-03-24T03:00:41.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.723+0000 7f7870d9e700 1 -- 172.21.15.17:0/3269364169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f786c1020e0 msgr2=0x7f786c198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:41.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.723+0000 7f7870d9e700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f786c1020e0 0x7f786c198230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f785c00e9f0 tx=0x7f785c00ed00 comp rx=0 tx=0).stop 2024-03-24T03:00:41.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.723+0000 7f7870d9e700 1 -- 172.21.15.17:0/3269364169 shutdown_connections 2024-03-24T03:00:41.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.723+0000 7f7870d9e700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f78580a2890 0x7f78580a4d50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:41.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.723+0000 7f7870d9e700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c103c80 0x7f786c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:41.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.723+0000 7f7870d9e700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f786c1020e0 0x7f786c198230 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:41.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.723+0000 7f7870d9e700 1 --2- 172.21.15.17:0/3269364169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f786c1032e0 0x7f786c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:41.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.723+0000 7f7870d9e700 1 -- 172.21.15.17:0/3269364169 >> 172.21.15.17:0/3269364169 conn(0x7f786c0fd6a0 msgr2=0x7f786c106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:41.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.723+0000 7f7870d9e700 1 -- 172.21.15.17:0/3269364169 shutdown_connections 2024-03-24T03:00:41.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:41.724+0000 7f7870d9e700 1 -- 172.21.15.17:0/3269364169 wait complete. 2024-03-24T03:00:41.727 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 28 2024-03-24T03:00:42.084 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:41 smithi017 ceph-mon[31926]: pgmap v358: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 7.3 KiB/s, 2 objects/s recovering 2024-03-24T03:00:42.084 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:41 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3269364169' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T03:00:42.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:41 smithi188 ceph-mon[36206]: pgmap v358: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 7.3 KiB/s, 2 objects/s recovering 2024-03-24T03:00:42.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:41 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3269364169' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T03:00:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:41 smithi019 ceph-mon[35944]: pgmap v358: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 7.3 KiB/s, 2 objects/s recovering 2024-03-24T03:00:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:41 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3269364169' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T03:00:42.452 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-24T03:00:42.703 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:00:44.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:43 smithi017 ceph-mon[31926]: pgmap v359: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 6.6 KiB/s, 1 objects/s recovering 2024-03-24T03:00:44.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:43 smithi188 ceph-mon[36206]: pgmap v359: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 6.6 KiB/s, 1 objects/s recovering 2024-03-24T03:00:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:43 smithi019 ceph-mon[35944]: pgmap v359: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 6.6 KiB/s, 1 objects/s recovering 2024-03-24T03:00:44.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.443+0000 7fb278f93700 1 -- 172.21.15.17:0/2199405649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2741021e0 msgr2=0x7fb274102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.443+0000 7fb278f93700 1 --2- 172.21.15.17:0/2199405649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2741021e0 0x7fb274102600 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb2680099c0 tx=0x7fb268009cd0 comp rx=0 tx=0).stop 2024-03-24T03:00:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.443+0000 7fb278f93700 1 -- 172.21.15.17:0/2199405649 shutdown_connections 2024-03-24T03:00:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.443+0000 7fb278f93700 1 --2- 172.21.15.17:0/2199405649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb274103d80 0x7fb27410a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.443+0000 7fb278f93700 1 --2- 172.21.15.17:0/2199405649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2741021e0 0x7fb274102600 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.443+0000 7fb278f93700 1 --2- 172.21.15.17:0/2199405649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2741033e0 0x7fb274103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.443+0000 7fb278f93700 1 -- 172.21.15.17:0/2199405649 >> 172.21.15.17:0/2199405649 conn(0x7fb2740fd7a0 msgr2=0x7fb2740ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.443+0000 7fb278f93700 1 -- 172.21.15.17:0/2199405649 shutdown_connections 2024-03-24T03:00:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.443+0000 7fb278f93700 1 -- 172.21.15.17:0/2199405649 wait complete. 2024-03-24T03:00:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.444+0000 7fb278f93700 1 Processor -- start 2024-03-24T03:00:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.444+0000 7fb278f93700 1 -- start start 2024-03-24T03:00:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.444+0000 7fb278f93700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2741021e0 0x7fb2741983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.444+0000 7fb278f93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2741033e0 0x7fb274198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.444+0000 7fb278f93700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb274103d80 0x7fb27419f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.444+0000 7fb278f93700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27410ddd0 con 0x7fb274103d80 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.444+0000 7fb278f93700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27410dc50 con 0x7fb2741033e0 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.444+0000 7fb278f93700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27410df50 con 0x7fb2741021e0 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.444+0000 7fb27259c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2741021e0 0x7fb2741983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.444+0000 7fb27259c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2741021e0 0x7fb2741983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:42630/0 (socket says 172.21.15.17:42630) 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.444+0000 7fb27259c700 1 -- 172.21.15.17:0/1013068224 learned_addr learned my addr 172.21.15.17:0/1013068224 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.444+0000 7fb271d9b700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2741033e0 0x7fb274198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb272d9d700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb274103d80 0x7fb27419f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb27259c700 1 -- 172.21.15.17:0/1013068224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2741033e0 msgr2=0x7fb274198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb27259c700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2741033e0 0x7fb274198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb27259c700 1 -- 172.21.15.17:0/1013068224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb274103d80 msgr2=0x7fb27419f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb27259c700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb274103d80 0x7fb27419f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb27259c700 1 -- 172.21.15.17:0/1013068224 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2741a0020 con 0x7fb2741021e0 2024-03-24T03:00:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb271d9b700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2741033e0 0x7fb274198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb27259c700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2741021e0 0x7fb2741983c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb26800f710 tx=0x7fb268009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb2637fe700 1 -- 172.21.15.17:0/1013068224 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268009e60 con 0x7fb2741021e0 2024-03-24T03:00:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb278f93700 1 -- 172.21.15.17:0/1013068224 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2741a02b0 con 0x7fb2741021e0 2024-03-24T03:00:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb2637fe700 1 -- 172.21.15.17:0/1013068224 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb268003960 con 0x7fb2741021e0 2024-03-24T03:00:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb2637fe700 1 -- 172.21.15.17:0/1013068224 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268005560 con 0x7fb2741021e0 2024-03-24T03:00:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.445+0000 7fb278f93700 1 -- 172.21.15.17:0/1013068224 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2741a07e0 con 0x7fb2741021e0 2024-03-24T03:00:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.446+0000 7fb278f93700 1 -- 172.21.15.17:0/1013068224 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb274068e50 con 0x7fb2741021e0 2024-03-24T03:00:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.448+0000 7fb2637fe700 1 -- 172.21.15.17:0/1013068224 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb268003b10 con 0x7fb2741021e0 2024-03-24T03:00:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.448+0000 7fb2637fe700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb25c0a2890 0x7fb25c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.448+0000 7fb271d9b700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb25c0a2890 0x7fb25c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.448+0000 7fb2637fe700 1 -- 172.21.15.17:0/1013068224 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fb2680cfce0 con 0x7fb2741021e0 2024-03-24T03:00:44.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.449+0000 7fb271d9b700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb25c0a2890 0x7fb25c0a4d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb264011490 tx=0x7fb264009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:44.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.451+0000 7fb2637fe700 1 -- 172.21.15.17:0/1013068224 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb268013070 con 0x7fb2741021e0 2024-03-24T03:00:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.656+0000 7fb278f93700 1 -- 172.21.15.17:0/1013068224 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fb27402da30 con 0x7fb2741021e0 2024-03-24T03:00:44.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.657+0000 7fb2637fe700 1 -- 172.21.15.17:0/1013068224 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) v1 ==== 56+0+5785 (secure 0 0 0) 0x7fb26809c970 con 0x7fb2741021e0 2024-03-24T03:00:44.657 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 93 2024-03-24T03:00:44.657 INFO:teuthology.orchestra.run.smithi017.stdout:fsid 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T03:00:44.657 INFO:teuthology.orchestra.run.smithi017.stdout:created 2024-03-24T02:50:35.152661+0000 2024-03-24T03:00:44.657 INFO:teuthology.orchestra.run.smithi017.stdout:modified 2024-03-24T03:00:31.817661+0000 2024-03-24T03:00:44.658 INFO:teuthology.orchestra.run.smithi017.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-24T03:00:44.658 INFO:teuthology.orchestra.run.smithi017.stdout:crush_version 27 2024-03-24T03:00:44.658 INFO:teuthology.orchestra.run.smithi017.stdout:full_ratio 0.95 2024-03-24T03:00:44.658 INFO:teuthology.orchestra.run.smithi017.stdout:backfillfull_ratio 0.9 2024-03-24T03:00:44.658 INFO:teuthology.orchestra.run.smithi017.stdout:nearfull_ratio 0.85 2024-03-24T03:00:44.658 INFO:teuthology.orchestra.run.smithi017.stdout:require_min_compat_client luminous 2024-03-24T03:00:44.658 INFO:teuthology.orchestra.run.smithi017.stdout:min_compat_client luminous 2024-03-24T03:00:44.658 INFO:teuthology.orchestra.run.smithi017.stdout:require_osd_release squid 2024-03-24T03:00:44.658 INFO:teuthology.orchestra.run.smithi017.stdout:stretch_mode_enabled false 2024-03-24T03:00:44.658 INFO:teuthology.orchestra.run.smithi017.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-24T03:00:44.658 INFO:teuthology.orchestra.run.smithi017.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:max_osd 12 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0 up in weight 1 up_from 8 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6802/3184461551,v1:172.21.15.17:6803/3184461551] [v2:172.21.15.17:6804/3184461551,v1:172.21.15.17:6805/3184461551] exists,up 07cdbc7a-3ba5-4284-8819-b9352e15e8c8 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1 up in weight 1 up_from 13 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6800/1476118517,v1:172.21.15.19:6801/1476118517] [v2:172.21.15.19:6802/1476118517,v1:172.21.15.19:6803/1476118517] exists,up 83235b08-6883-4443-8f53-7cd7f3db924f 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6800/2030207482,v1:172.21.15.188:6801/2030207482] [v2:172.21.15.188:6802/2030207482,v1:172.21.15.188:6803/2030207482] exists,up 9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3 up in weight 1 up_from 25 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6810/1744816113,v1:172.21.15.17:6811/1744816113] [v2:172.21.15.17:6812/1744816113,v1:172.21.15.17:6813/1744816113] exists,up 1ba93700-990f-4c99-9d48-1adf99322cf8 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6808/4029793962,v1:172.21.15.19:6809/4029793962] [v2:172.21.15.19:6810/4029793962,v1:172.21.15.19:6811/4029793962] exists,up 659ccfee-f6c0-4d35-b557-f7978d553346 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6808/2503049360,v1:172.21.15.188:6809/2503049360] [v2:172.21.15.188:6810/2503049360,v1:172.21.15.188:6811/2503049360] exists,up a3ec6587-2c61-4188-b4af-9f36ad6c4abc 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6818/3664721354,v1:172.21.15.17:6819/3664721354] [v2:172.21.15.17:6820/3664721354,v1:172.21.15.17:6821/3664721354] exists,up 6713217d-57aa-4293-9cb7-de935d0a8537 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6816/2743063507,v1:172.21.15.19:6817/2743063507] [v2:172.21.15.19:6818/2743063507,v1:172.21.15.19:6819/2743063507] exists,up 4aa6635c-d640-46d9-9124-3e7afe24e65d 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:osd.8 up in weight 1 up_from 59 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6816/1144752279,v1:172.21.15.188:6817/1144752279] [v2:172.21.15.188:6818/1144752279,v1:172.21.15.188:6819/1144752279] exists,up 9df9a933-c0be-4277-9837-95fb58c813d6 2024-03-24T03:00:44.659 INFO:teuthology.orchestra.run.smithi017.stdout:osd.9 up in weight 1 up_from 67 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6826/116991070,v1:172.21.15.17:6827/116991070] [v2:172.21.15.17:6828/116991070,v1:172.21.15.17:6829/116991070] exists,up 50a0ca53-b96d-47ed-84f6-b7fc6a75c23f 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:osd.10 up in weight 1 up_from 72 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6824/3678221892,v1:172.21.15.19:6825/3678221892] [v2:172.21.15.19:6826/3678221892,v1:172.21.15.19:6827/3678221892] exists,up e702b87a-9535-42db-aa8f-58f35cecb5d9 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6824/2579389713,v1:172.21.15.188:6825/2579389713] [v2:172.21.15.188:6826/2579389713,v1:172.21.15.188:6827/2579389713] exists,up bb1fc489-8371-4d59-9086-2629adec0374 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 2.8 [2,11] 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 2.b [2,4,7,6] 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 2.14 [2,4] 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 2.18 [5,11] 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 2.1a [8,1] 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 2.1f [8,1] 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.17 [4,7,5,8] 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.1e [3,9] 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.21 [4,8] 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.23 [4,7] 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.27 [4,6] 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.35 [1,9] 2024-03-24T03:00:44.660 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.38 [3,9,5,6] 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 4.b [9,8] 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 4.11 [5,8] 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 4.17 [9,10] 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 4.1c [9,8] 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/1870031079 expires 2024-03-25T02:51:29.864563+0000 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:6801/3007657427 expires 2024-03-25T02:51:29.864563+0000 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:6800/3007657427 expires 2024-03-25T02:51:29.864563+0000 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/2226886960 expires 2024-03-25T02:51:02.292414+0000 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:6801/4291784529 expires 2024-03-25T02:51:02.292414+0000 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/4202143244 expires 2024-03-25T02:51:02.292414+0000 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:6800/4291784529 expires 2024-03-25T02:51:02.292414+0000 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/630595937 expires 2024-03-25T02:51:02.292414+0000 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/532405 expires 2024-03-25T02:51:29.864563+0000 2024-03-24T03:00:44.661 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/467902990 expires 2024-03-25T02:51:29.864563+0000 2024-03-24T03:00:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.659+0000 7fb278f93700 1 -- 172.21.15.17:0/1013068224 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb25c0a2890 msgr2=0x7fb25c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.659+0000 7fb278f93700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb25c0a2890 0x7fb25c0a4d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb264011490 tx=0x7fb264009260 comp rx=0 tx=0).stop 2024-03-24T03:00:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.659+0000 7fb278f93700 1 -- 172.21.15.17:0/1013068224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2741021e0 msgr2=0x7fb2741983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.659+0000 7fb278f93700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2741021e0 0x7fb2741983c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb26800f710 tx=0x7fb268009580 comp rx=0 tx=0).stop 2024-03-24T03:00:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.659+0000 7fb278f93700 1 -- 172.21.15.17:0/1013068224 shutdown_connections 2024-03-24T03:00:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.659+0000 7fb278f93700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb25c0a2890 0x7fb25c0a4d50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.659+0000 7fb278f93700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb274103d80 0x7fb27419f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.659+0000 7fb278f93700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2741021e0 0x7fb2741983c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.659+0000 7fb278f93700 1 --2- 172.21.15.17:0/1013068224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2741033e0 0x7fb274198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.659+0000 7fb278f93700 1 -- 172.21.15.17:0/1013068224 >> 172.21.15.17:0/1013068224 conn(0x7fb2740fd7a0 msgr2=0x7fb274106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.659+0000 7fb278f93700 1 -- 172.21.15.17:0/1013068224 shutdown_connections 2024-03-24T03:00:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:44.659+0000 7fb278f93700 1 -- 172.21.15.17:0/1013068224 wait complete. 2024-03-24T03:00:44.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:44 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1013068224' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-24T03:00:45.198 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:44 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1013068224' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-24T03:00:45.221 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:44 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1013068224' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-24T03:00:45.280 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-24T03:00:45.292 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T03:00:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.399+0000 7fb397ef8640 1 -- 172.21.15.17:0/2900794024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb390102170 msgr2=0x7fb390108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.399+0000 7fb397ef8640 1 --2- 172.21.15.17:0/2900794024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb390102170 0x7fb390108a00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb388009ca0 tx=0x7fb38802c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.399+0000 7fb397ef8640 1 -- 172.21.15.17:0/2900794024 shutdown_connections 2024-03-24T03:00:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.399+0000 7fb397ef8640 1 --2- 172.21.15.17:0/2900794024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb390102170 0x7fb390108a00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.399+0000 7fb397ef8640 1 --2- 172.21.15.17:0/2900794024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3901017b0 0x7fb390101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.399+0000 7fb397ef8640 1 --2- 172.21.15.17:0/2900794024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3901005c0 0x7fb3901009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.399+0000 7fb397ef8640 1 -- 172.21.15.17:0/2900794024 >> 172.21.15.17:0/2900794024 conn(0x7fb3900fbb30 msgr2=0x7fb3900fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.399+0000 7fb397ef8640 1 -- 172.21.15.17:0/2900794024 shutdown_connections 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.399+0000 7fb397ef8640 1 -- 172.21.15.17:0/2900794024 wait complete. 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb397ef8640 1 Processor -- start 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb397ef8640 1 -- start start 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb397ef8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3901005c0 0x7fb3901968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb397ef8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3901017b0 0x7fb390196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb397ef8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb390102170 0x7fb39019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb397ef8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39010c240 con 0x7fb390102170 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb397ef8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39010c0c0 con 0x7fb3901017b0 2024-03-24T03:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb397ef8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39010c3c0 con 0x7fb3901005c0 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb396495640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb390102170 0x7fb39019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb396495640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb390102170 0x7fb39019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41604/0 (socket says 172.21.15.17:41604) 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb395c94640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3901005c0 0x7fb3901968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb396495640 1 -- 172.21.15.17:0/3467619822 learned_addr learned my addr 172.21.15.17:0/3467619822 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb395493640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3901017b0 0x7fb390196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb396495640 1 -- 172.21.15.17:0/3467619822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3901005c0 msgr2=0x7fb3901968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb396495640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3901005c0 0x7fb3901968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb396495640 1 -- 172.21.15.17:0/3467619822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3901017b0 msgr2=0x7fb390196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb396495640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3901017b0 0x7fb390196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb396495640 1 -- 172.21.15.17:0/3467619822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39019e4d0 con 0x7fb390102170 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.400+0000 7fb396495640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb390102170 0x7fb39019de60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb388037040 tx=0x7fb388032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.401+0000 7fb386ffd640 1 -- 172.21.15.17:0/3467619822 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388041020 con 0x7fb390102170 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.401+0000 7fb386ffd640 1 -- 172.21.15.17:0/3467619822 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb388034cd0 con 0x7fb390102170 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.401+0000 7fb397ef8640 1 -- 172.21.15.17:0/3467619822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39019e760 con 0x7fb390102170 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.401+0000 7fb386ffd640 1 -- 172.21.15.17:0/3467619822 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388039040 con 0x7fb390102170 2024-03-24T03:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.401+0000 7fb397ef8640 1 -- 172.21.15.17:0/3467619822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39019ec50 con 0x7fb390102170 2024-03-24T03:00:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.402+0000 7fb397ef8640 1 -- 172.21.15.17:0/3467619822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3901078e0 con 0x7fb390102170 2024-03-24T03:00:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.403+0000 7fb386ffd640 1 -- 172.21.15.17:0/3467619822 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb388044020 con 0x7fb390102170 2024-03-24T03:00:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.403+0000 7fb386ffd640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb37c0a2800 0x7fb37c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.403+0000 7fb395c94640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb37c0a2800 0x7fb37c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.403+0000 7fb386ffd640 1 -- 172.21.15.17:0/3467619822 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fb3880f1180 con 0x7fb390102170 2024-03-24T03:00:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.404+0000 7fb395c94640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb37c0a2800 0x7fb37c0a4cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb390101610 tx=0x7fb38c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:45.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.407+0000 7fb386ffd640 1 -- 172.21.15.17:0/3467619822 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb38804b020 con 0x7fb390102170 2024-03-24T03:00:45.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.590+0000 7fb397ef8640 1 -- 172.21.15.17:0/3467619822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb39002d240 con 0x7fb390102170 2024-03-24T03:00:45.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.592+0000 7fb386ffd640 1 -- 172.21.15.17:0/3467619822 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24955 (secure 0 0 0) 0x7fb3880bde70 con 0x7fb390102170 2024-03-24T03:00:45.593 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:45.593 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":93,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T03:00:31.817661+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T03:00:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.594+0000 7fb397ef8640 1 -- 172.21.15.17:0/3467619822 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb37c0a2800 msgr2=0x7fb37c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.594+0000 7fb397ef8640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb37c0a2800 0x7fb37c0a4cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb390101610 tx=0x7fb38c005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.594+0000 7fb397ef8640 1 -- 172.21.15.17:0/3467619822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb390102170 msgr2=0x7fb39019de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.594+0000 7fb397ef8640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb390102170 0x7fb39019de60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb388037040 tx=0x7fb388032f40 comp rx=0 tx=0).stop 2024-03-24T03:00:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.594+0000 7fb397ef8640 1 -- 172.21.15.17:0/3467619822 shutdown_connections 2024-03-24T03:00:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.594+0000 7fb397ef8640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fb37c0a2800 0x7fb37c0a4cc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.594+0000 7fb397ef8640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb390102170 0x7fb39019de60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.594+0000 7fb397ef8640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3901017b0 0x7fb390196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.594+0000 7fb397ef8640 1 --2- 172.21.15.17:0/3467619822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3901005c0 0x7fb3901968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.594+0000 7fb397ef8640 1 -- 172.21.15.17:0/3467619822 >> 172.21.15.17:0/3467619822 conn(0x7fb3900fbb30 msgr2=0x7fb390104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.594+0000 7fb397ef8640 1 -- 172.21.15.17:0/3467619822 shutdown_connections 2024-03-24T03:00:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.594+0000 7fb397ef8640 1 -- 172.21.15.17:0/3467619822 wait complete. 2024-03-24T03:00:45.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T02:53:54.207139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-24T02:58:34.785192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-24T02:58:35.679772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-24T02:58:39.221772+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-24T03:00:45.607 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T03:00:45.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.712+0000 7fa02b1b9640 1 -- 172.21.15.17:0/3038812699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0241005b0 msgr2=0x7fa0241009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.712+0000 7fa02b1b9640 1 --2- 172.21.15.17:0/3038812699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0241005b0 0x7fa0241009d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa018009900 tx=0x7fa01802bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.712+0000 7fa02b1b9640 1 -- 172.21.15.17:0/3038812699 shutdown_connections 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.712+0000 7fa02b1b9640 1 --2- 172.21.15.17:0/3038812699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa024102160 0x7fa0241089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.712+0000 7fa02b1b9640 1 --2- 172.21.15.17:0/3038812699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0241017a0 0x7fa024101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.712+0000 7fa02b1b9640 1 --2- 172.21.15.17:0/3038812699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0241005b0 0x7fa0241009d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.712+0000 7fa02b1b9640 1 -- 172.21.15.17:0/3038812699 >> 172.21.15.17:0/3038812699 conn(0x7fa0240fbb40 msgr2=0x7fa0240fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.712+0000 7fa02b1b9640 1 -- 172.21.15.17:0/3038812699 shutdown_connections 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.712+0000 7fa02b1b9640 1 -- 172.21.15.17:0/3038812699 wait complete. 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.712+0000 7fa02b1b9640 1 Processor -- start 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa02b1b9640 1 -- start start 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa02b1b9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0241005b0 0x7fa024104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa02b1b9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0241017a0 0x7fa024102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa02b1b9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa024102160 0x7fa024103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa02b1b9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02410c220 con 0x7fa0241005b0 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa02b1b9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02410c0a0 con 0x7fa0241017a0 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa02b1b9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02410c3a0 con 0x7fa024102160 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa028f55640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0241005b0 0x7fa024104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa028f55640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0241005b0 0x7fa024104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41626/0 (socket says 172.21.15.17:41626) 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa028f55640 1 -- 172.21.15.17:0/2639292344 learned_addr learned my addr 172.21.15.17:0/2639292344 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa023fff640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0241017a0 0x7fa024102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa029756640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa024102160 0x7fa024103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa028f55640 1 -- 172.21.15.17:0/2639292344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa024102160 msgr2=0x7fa024103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa028f55640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa024102160 0x7fa024103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa028f55640 1 -- 172.21.15.17:0/2639292344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0241017a0 msgr2=0x7fa024102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa028f55640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0241017a0 0x7fa024102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa028f55640 1 -- 172.21.15.17:0/2639292344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa024103aa0 con 0x7fa0241005b0 2024-03-24T03:00:45.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa023fff640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0241017a0 0x7fa024102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:45.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa028f55640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0241005b0 0x7fa024104690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa018000c00 tx=0x7fa01802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:45.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.713+0000 7fa021ffb640 1 -- 172.21.15.17:0/2639292344 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018041070 con 0x7fa0241005b0 2024-03-24T03:00:45.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.714+0000 7fa021ffb640 1 -- 172.21.15.17:0/2639292344 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa018004450 con 0x7fa0241005b0 2024-03-24T03:00:45.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.714+0000 7fa021ffb640 1 -- 172.21.15.17:0/2639292344 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018034440 con 0x7fa0241005b0 2024-03-24T03:00:45.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.714+0000 7fa02b1b9640 1 -- 172.21.15.17:0/2639292344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0241a1aa0 con 0x7fa0241005b0 2024-03-24T03:00:45.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.714+0000 7fa02b1b9640 1 -- 172.21.15.17:0/2639292344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0241a1ff0 con 0x7fa0241005b0 2024-03-24T03:00:45.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.715+0000 7fa02b1b9640 1 -- 172.21.15.17:0/2639292344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0241a21c0 con 0x7fa0241005b0 2024-03-24T03:00:45.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.716+0000 7fa021ffb640 1 -- 172.21.15.17:0/2639292344 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa018044070 con 0x7fa0241005b0 2024-03-24T03:00:45.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.716+0000 7fa021ffb640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa0140a2800 0x7fa0140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:45.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.716+0000 7fa023fff640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa0140a2800 0x7fa0140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:45.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.717+0000 7fa021ffb640 1 -- 172.21.15.17:0/2639292344 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fa0180f0bd0 con 0x7fa0241005b0 2024-03-24T03:00:45.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.717+0000 7fa023fff640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa0140a2800 0x7fa0140a4cc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa024104400 tx=0x7fa010005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:45.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.720+0000 7fa021ffb640 1 -- 172.21.15.17:0/2639292344 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa0180bd210 con 0x7fa0241005b0 2024-03-24T03:00:45.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.904+0000 7fa02b1b9640 1 -- 172.21.15.17:0/2639292344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0241a2390 con 0x7fa0241005b0 2024-03-24T03:00:45.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.904+0000 7fa021ffb640 1 -- 172.21.15.17:0/2639292344 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0180c12b0 con 0x7fa0241005b0 2024-03-24T03:00:45.904 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-24T03:00:45.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.906+0000 7fa02b1b9640 1 -- 172.21.15.17:0/2639292344 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa0140a2800 msgr2=0x7fa0140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:45.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.906+0000 7fa02b1b9640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa0140a2800 0x7fa0140a4cc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa024104400 tx=0x7fa010005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:45.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.906+0000 7fa02b1b9640 1 -- 172.21.15.17:0/2639292344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0241005b0 msgr2=0x7fa024104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:45.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.906+0000 7fa02b1b9640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0241005b0 0x7fa024104690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa018000c00 tx=0x7fa01802c640 comp rx=0 tx=0).stop 2024-03-24T03:00:45.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.906+0000 7fa02b1b9640 1 -- 172.21.15.17:0/2639292344 shutdown_connections 2024-03-24T03:00:45.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.906+0000 7fa02b1b9640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa0140a2800 0x7fa0140a4cc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.906+0000 7fa02b1b9640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa024102160 0x7fa024103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.906+0000 7fa02b1b9640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0241017a0 0x7fa024102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.906+0000 7fa02b1b9640 1 --2- 172.21.15.17:0/2639292344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0241005b0 0x7fa024104690 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:45.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.906+0000 7fa02b1b9640 1 -- 172.21.15.17:0/2639292344 >> 172.21.15.17:0/2639292344 conn(0x7fa0240fbb40 msgr2=0x7fa024069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:45.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.906+0000 7fa02b1b9640 1 -- 172.21.15.17:0/2639292344 shutdown_connections 2024-03-24T03:00:45.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:45.906+0000 7fa02b1b9640 1 -- 172.21.15.17:0/2639292344 wait complete. 2024-03-24T03:00:45.917 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-24T03:00:46.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.022+0000 7fca27f16640 1 -- 172.21.15.17:0/3140867723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca201005a0 msgr2=0x7fca201009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.022+0000 7fca27f16640 1 --2- 172.21.15.17:0/3140867723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca201005a0 0x7fca201009c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009900 tx=0x7fca1c02bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:46.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.022+0000 7fca27f16640 1 -- 172.21.15.17:0/3140867723 shutdown_connections 2024-03-24T03:00:46.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.022+0000 7fca27f16640 1 --2- 172.21.15.17:0/3140867723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca20102150 0x7fca201089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.022+0000 7fca27f16640 1 --2- 172.21.15.17:0/3140867723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca20101790 0x7fca20101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.022+0000 7fca27f16640 1 --2- 172.21.15.17:0/3140867723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca201005a0 0x7fca201009c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.022+0000 7fca27f16640 1 -- 172.21.15.17:0/3140867723 >> 172.21.15.17:0/3140867723 conn(0x7fca200fbb30 msgr2=0x7fca200fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:46.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca27f16640 1 -- 172.21.15.17:0/3140867723 shutdown_connections 2024-03-24T03:00:46.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca27f16640 1 -- 172.21.15.17:0/3140867723 wait complete. 2024-03-24T03:00:46.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca27f16640 1 Processor -- start 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca27f16640 1 -- start start 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca27f16640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca201005a0 0x7fca201979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca27f16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca20101790 0x7fca20197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca27f16640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca20102150 0x7fca2019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca27f16640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2010d340 con 0x7fca20102150 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca27f16640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2010d1c0 con 0x7fca20101790 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca27f16640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2010d4c0 con 0x7fca201005a0 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca264b3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca20102150 0x7fca2019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca264b3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca20102150 0x7fca2019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41656/0 (socket says 172.21.15.17:41656) 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca264b3640 1 -- 172.21.15.17:0/544048140 learned_addr learned my addr 172.21.15.17:0/544048140 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca25cb2640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca201005a0 0x7fca201979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.023+0000 7fca254b1640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca20101790 0x7fca20197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.024+0000 7fca264b3640 1 -- 172.21.15.17:0/544048140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca201005a0 msgr2=0x7fca201979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.024+0000 7fca264b3640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca201005a0 0x7fca201979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.024+0000 7fca264b3640 1 -- 172.21.15.17:0/544048140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca20101790 msgr2=0x7fca20197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.024+0000 7fca264b3640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca20101790 0x7fca20197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.024+0000 7fca264b3640 1 -- 172.21.15.17:0/544048140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2019f600 con 0x7fca20102150 2024-03-24T03:00:46.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.024+0000 7fca25cb2640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca201005a0 0x7fca201979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:46.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.024+0000 7fca264b3640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca20102150 0x7fca2019ef90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fca1800eb10 tx=0x7fca1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:46.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.024+0000 7fca16ffd640 1 -- 172.21.15.17:0/544048140 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18004b20 con 0x7fca20102150 2024-03-24T03:00:46.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.024+0000 7fca16ffd640 1 -- 172.21.15.17:0/544048140 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca18004cc0 con 0x7fca20102150 2024-03-24T03:00:46.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.024+0000 7fca27f16640 1 -- 172.21.15.17:0/544048140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2019f8f0 con 0x7fca20102150 2024-03-24T03:00:46.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.024+0000 7fca16ffd640 1 -- 172.21.15.17:0/544048140 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18010760 con 0x7fca20102150 2024-03-24T03:00:46.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.024+0000 7fca27f16640 1 -- 172.21.15.17:0/544048140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2019fdf0 con 0x7fca20102150 2024-03-24T03:00:46.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.025+0000 7fca27f16640 1 -- 172.21.15.17:0/544048140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca20107940 con 0x7fca20102150 2024-03-24T03:00:46.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.026+0000 7fca16ffd640 1 -- 172.21.15.17:0/544048140 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fca18005020 con 0x7fca20102150 2024-03-24T03:00:46.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.026+0000 7fca16ffd640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fca0c0a2800 0x7fca0c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.027+0000 7fca25cb2640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fca0c0a2800 0x7fca0c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.027+0000 7fca16ffd640 1 -- 172.21.15.17:0/544048140 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fca180cf410 con 0x7fca20102150 2024-03-24T03:00:46.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.027+0000 7fca25cb2640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fca0c0a2800 0x7fca0c0a4cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fca1c000c00 tx=0x7fca1c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:46.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.031+0000 7fca16ffd640 1 -- 172.21.15.17:0/544048140 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fca1809c070 con 0x7fca20102150 2024-03-24T03:00:46.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:45 smithi017 ceph-mon[31926]: pgmap v360: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 5.6 KiB/s, 1 objects/s recovering 2024-03-24T03:00:46.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:45 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3467619822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:46.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.214+0000 7fca27f16640 1 -- 172.21.15.17:0/544048140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca200622a0 con 0x7fca20102150 2024-03-24T03:00:46.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.214+0000 7fca16ffd640 1 -- 172.21.15.17:0/544048140 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fca1809b970 con 0x7fca20102150 2024-03-24T03:00:46.214 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T03:00:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.216+0000 7fca27f16640 1 -- 172.21.15.17:0/544048140 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fca0c0a2800 msgr2=0x7fca0c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.216+0000 7fca27f16640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fca0c0a2800 0x7fca0c0a4cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fca1c000c00 tx=0x7fca1c005fd0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.216+0000 7fca27f16640 1 -- 172.21.15.17:0/544048140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca20102150 msgr2=0x7fca2019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.216+0000 7fca27f16640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca20102150 0x7fca2019ef90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fca1800eb10 tx=0x7fca1800c2d0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.216+0000 7fca27f16640 1 -- 172.21.15.17:0/544048140 shutdown_connections 2024-03-24T03:00:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.216+0000 7fca27f16640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fca0c0a2800 0x7fca0c0a4cc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.216+0000 7fca27f16640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca20102150 0x7fca2019ef90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.216+0000 7fca27f16640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca20101790 0x7fca20197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.216+0000 7fca27f16640 1 --2- 172.21.15.17:0/544048140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca201005a0 0x7fca201979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.216+0000 7fca27f16640 1 -- 172.21.15.17:0/544048140 >> 172.21.15.17:0/544048140 conn(0x7fca200fbb30 msgr2=0x7fca20104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.216+0000 7fca27f16640 1 -- 172.21.15.17:0/544048140 shutdown_connections 2024-03-24T03:00:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.217+0000 7fca27f16640 1 -- 172.21.15.17:0/544048140 wait complete. 2024-03-24T03:00:46.228 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-24T03:00:46.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:45 smithi188 ceph-mon[36206]: pgmap v360: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 5.6 KiB/s, 1 objects/s recovering 2024-03-24T03:00:46.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:45 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3467619822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:46.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:45 smithi019 ceph-mon[35944]: pgmap v360: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 5.6 KiB/s, 1 objects/s recovering 2024-03-24T03:00:46.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:45 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3467619822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:46.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.333+0000 7fc961827640 1 -- 172.21.15.17:0/2044924731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95c102140 msgr2=0x7fc95c104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.333+0000 7fc961827640 1 --2- 172.21.15.17:0/2044924731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95c102140 0x7fc95c104530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009860 tx=0x7fc94c02bb70 comp rx=0 tx=0).stop 2024-03-24T03:00:46.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.333+0000 7fc961827640 1 -- 172.21.15.17:0/2044924731 shutdown_connections 2024-03-24T03:00:46.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.333+0000 7fc961827640 1 --2- 172.21.15.17:0/2044924731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc95c1073a0 0x7fc95c1097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.333+0000 7fc961827640 1 --2- 172.21.15.17:0/2044924731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95c104a70 0x7fc95c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.333+0000 7fc961827640 1 --2- 172.21.15.17:0/2044924731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95c102140 0x7fc95c104530 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.333+0000 7fc961827640 1 -- 172.21.15.17:0/2044924731 >> 172.21.15.17:0/2044924731 conn(0x7fc95c0fbb50 msgr2=0x7fc95c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:46.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.333+0000 7fc961827640 1 -- 172.21.15.17:0/2044924731 shutdown_connections 2024-03-24T03:00:46.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.333+0000 7fc961827640 1 -- 172.21.15.17:0/2044924731 wait complete. 2024-03-24T03:00:46.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.333+0000 7fc961827640 1 Processor -- start 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc961827640 1 -- start start 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc961827640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95c102140 0x7fc95c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc961827640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc95c104a70 0x7fc95c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc961827640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95c1073a0 0x7fc95c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc961827640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c10d350 con 0x7fc95c104a70 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc961827640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c10d1d0 con 0x7fc95c1073a0 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc961827640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c10d4d0 con 0x7fc95c102140 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc9527fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc95c104a70 0x7fc95c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc9527fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc95c104a70 0x7fc95c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41684/0 (socket says 172.21.15.17:41684) 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc9527fc640 1 -- 172.21.15.17:0/1927939564 learned_addr learned my addr 172.21.15.17:0/1927939564 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc95affd640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95c102140 0x7fc95c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc95b7fe640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95c1073a0 0x7fc95c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc9527fc640 1 -- 172.21.15.17:0/1927939564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95c102140 msgr2=0x7fc95c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc9527fc640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95c102140 0x7fc95c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc9527fc640 1 -- 172.21.15.17:0/1927939564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95c1073a0 msgr2=0x7fc95c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc9527fc640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95c1073a0 0x7fc95c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc9527fc640 1 -- 172.21.15.17:0/1927939564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c19f640 con 0x7fc95c104a70 2024-03-24T03:00:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.334+0000 7fc95affd640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95c102140 0x7fc95c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.335+0000 7fc9527fc640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc95c104a70 0x7fc95c197f60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc94400e9a0 tx=0x7fc94400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.335+0000 7fc960825640 1 -- 172.21.15.17:0/1927939564 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94400cd80 con 0x7fc95c104a70 2024-03-24T03:00:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.335+0000 7fc961827640 1 -- 172.21.15.17:0/1927939564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95c19f930 con 0x7fc95c104a70 2024-03-24T03:00:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.335+0000 7fc960825640 1 -- 172.21.15.17:0/1927939564 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9440035f0 con 0x7fc95c104a70 2024-03-24T03:00:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.335+0000 7fc960825640 1 -- 172.21.15.17:0/1927939564 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944010670 con 0x7fc95c104a70 2024-03-24T03:00:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.335+0000 7fc961827640 1 -- 172.21.15.17:0/1927939564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95c19fe80 con 0x7fc95c104a70 2024-03-24T03:00:46.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.336+0000 7fc961827640 1 -- 172.21.15.17:0/1927939564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95c0fd640 con 0x7fc95c104a70 2024-03-24T03:00:46.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.338+0000 7fc960825640 1 -- 172.21.15.17:0/1927939564 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc9440037a0 con 0x7fc95c104a70 2024-03-24T03:00:46.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.338+0000 7fc960825640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc9480a2800 0x7fc9480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.338+0000 7fc960825640 1 -- 172.21.15.17:0/1927939564 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fc9440cf1e0 con 0x7fc95c104a70 2024-03-24T03:00:46.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.338+0000 7fc95affd640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc9480a2800 0x7fc9480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.339+0000 7fc95affd640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc9480a2800 0x7fc9480a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc94c02c180 tx=0x7fc94c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:46.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.342+0000 7fc960825640 1 -- 172.21.15.17:0/1927939564 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc94409bed0 con 0x7fc95c104a70 2024-03-24T03:00:46.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.526+0000 7fc961827640 1 -- 172.21.15.17:0/1927939564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc95c02d270 con 0x7fc95c104a70 2024-03-24T03:00:46.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.526+0000 7fc960825640 1 -- 172.21.15.17:0/1927939564 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc94409b7d0 con 0x7fc95c104a70 2024-03-24T03:00:46.526 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-24T03:00:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.528+0000 7fc961827640 1 -- 172.21.15.17:0/1927939564 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc9480a2800 msgr2=0x7fc9480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.528+0000 7fc961827640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc9480a2800 0x7fc9480a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc94c02c180 tx=0x7fc94c02e040 comp rx=0 tx=0).stop 2024-03-24T03:00:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.528+0000 7fc961827640 1 -- 172.21.15.17:0/1927939564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc95c104a70 msgr2=0x7fc95c197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.528+0000 7fc961827640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc95c104a70 0x7fc95c197f60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc94400e9a0 tx=0x7fc94400ee60 comp rx=0 tx=0).stop 2024-03-24T03:00:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.528+0000 7fc961827640 1 -- 172.21.15.17:0/1927939564 shutdown_connections 2024-03-24T03:00:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.528+0000 7fc961827640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc9480a2800 0x7fc9480a4cc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.528+0000 7fc961827640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95c1073a0 0x7fc95c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.528+0000 7fc961827640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc95c104a70 0x7fc95c197f60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.528+0000 7fc961827640 1 --2- 172.21.15.17:0/1927939564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95c102140 0x7fc95c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.528+0000 7fc961827640 1 -- 172.21.15.17:0/1927939564 >> 172.21.15.17:0/1927939564 conn(0x7fc95c0fbb50 msgr2=0x7fc95c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.528+0000 7fc961827640 1 -- 172.21.15.17:0/1927939564 shutdown_connections 2024-03-24T03:00:46.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.528+0000 7fc961827640 1 -- 172.21.15.17:0/1927939564 wait complete. 2024-03-24T03:00:46.540 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-24T03:00:46.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 -- 172.21.15.17:0/2895150414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfc81028c0 msgr2=0x7fdfc8102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 --2- 172.21.15.17:0/2895150414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfc81028c0 0x7fdfc8102d40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009900 tx=0x7fdfb802bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 -- 172.21.15.17:0/2895150414 shutdown_connections 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 --2- 172.21.15.17:0/2895150414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc8103280 0x7fdfc8109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 --2- 172.21.15.17:0/2895150414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfc81028c0 0x7fdfc8102d40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 --2- 172.21.15.17:0/2895150414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81016d0 0x7fdfc8101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 -- 172.21.15.17:0/2895150414 >> 172.21.15.17:0/2895150414 conn(0x7fdfc80fcc40 msgr2=0x7fdfc80ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 -- 172.21.15.17:0/2895150414 shutdown_connections 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 -- 172.21.15.17:0/2895150414 wait complete. 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 Processor -- start 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 -- start start 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81016d0 0x7fdfc81979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.646+0000 7fdfcff87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc81028c0 0x7fdfc8197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfcff87640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfc8103280 0x7fdfc819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfcff87640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc810d380 con 0x7fdfc8103280 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfcff87640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc810d200 con 0x7fdfc81028c0 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfcff87640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc810d500 con 0x7fdfc81016d0 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfce524640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfc8103280 0x7fdfc819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfce524640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfc8103280 0x7fdfc819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41704/0 (socket says 172.21.15.17:41704) 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfcdd23640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81016d0 0x7fdfc81979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfce524640 1 -- 172.21.15.17:0/229217832 learned_addr learned my addr 172.21.15.17:0/229217832 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfcd522640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc81028c0 0x7fdfc8197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfce524640 1 -- 172.21.15.17:0/229217832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81016d0 msgr2=0x7fdfc81979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfce524640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81016d0 0x7fdfc81979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfce524640 1 -- 172.21.15.17:0/229217832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc81028c0 msgr2=0x7fdfc8197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfce524640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc81028c0 0x7fdfc8197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfce524640 1 -- 172.21.15.17:0/229217832 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc819f5f0 con 0x7fdfc8103280 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfcdd23640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81016d0 0x7fdfc81979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfce524640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfc8103280 0x7fdfc819ef80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400eb10 tx=0x7fdfc400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:46.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfbeffd640 1 -- 172.21.15.17:0/229217832 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4004b20 con 0x7fdfc8103280 2024-03-24T03:00:46.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfbeffd640 1 -- 172.21.15.17:0/229217832 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfc4004cc0 con 0x7fdfc8103280 2024-03-24T03:00:46.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfcff87640 1 -- 172.21.15.17:0/229217832 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc819f8e0 con 0x7fdfc8103280 2024-03-24T03:00:46.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfbeffd640 1 -- 172.21.15.17:0/229217832 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4010760 con 0x7fdfc8103280 2024-03-24T03:00:46.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.647+0000 7fdfcff87640 1 -- 172.21.15.17:0/229217832 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc819fdb0 con 0x7fdfc8103280 2024-03-24T03:00:46.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.648+0000 7fdfcff87640 1 -- 172.21.15.17:0/229217832 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc81089f0 con 0x7fdfc8103280 2024-03-24T03:00:46.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.649+0000 7fdfbeffd640 1 -- 172.21.15.17:0/229217832 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdfc4005000 con 0x7fdfc8103280 2024-03-24T03:00:46.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.650+0000 7fdfbeffd640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdfb40a2800 0x7fdfb40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.650+0000 7fdfcdd23640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdfb40a2800 0x7fdfb40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.650+0000 7fdfbeffd640 1 -- 172.21.15.17:0/229217832 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fdfc40cf6a0 con 0x7fdfc8103280 2024-03-24T03:00:46.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.650+0000 7fdfcdd23640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdfb40a2800 0x7fdfb40a4cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8102720 tx=0x7fdfc0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:46.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.653+0000 7fdfbeffd640 1 -- 172.21.15.17:0/229217832 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdfc4053740 con 0x7fdfc8103280 2024-03-24T03:00:46.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.834+0000 7fdfcff87640 1 -- 172.21.15.17:0/229217832 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdfc802d200 con 0x7fdfc8103280 2024-03-24T03:00:46.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.834+0000 7fdfbeffd640 1 -- 172.21.15.17:0/229217832 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdfc409c2a0 con 0x7fdfc8103280 2024-03-24T03:00:46.835 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T03:00:46.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.836+0000 7fdfcff87640 1 -- 172.21.15.17:0/229217832 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdfb40a2800 msgr2=0x7fdfb40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.836+0000 7fdfcff87640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdfb40a2800 0x7fdfb40a4cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8102720 tx=0x7fdfc0005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.836+0000 7fdfcff87640 1 -- 172.21.15.17:0/229217832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfc8103280 msgr2=0x7fdfc819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.836+0000 7fdfcff87640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfc8103280 0x7fdfc819ef80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400eb10 tx=0x7fdfc400c2d0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.836+0000 7fdfcff87640 1 -- 172.21.15.17:0/229217832 shutdown_connections 2024-03-24T03:00:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.836+0000 7fdfcff87640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fdfb40a2800 0x7fdfb40a4cc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.836+0000 7fdfcff87640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfc8103280 0x7fdfc819ef80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.836+0000 7fdfcff87640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc81028c0 0x7fdfc8197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.836+0000 7fdfcff87640 1 --2- 172.21.15.17:0/229217832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81016d0 0x7fdfc81979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.836+0000 7fdfcff87640 1 -- 172.21.15.17:0/229217832 >> 172.21.15.17:0/229217832 conn(0x7fdfc80fcc40 msgr2=0x7fdfc8105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.836+0000 7fdfcff87640 1 -- 172.21.15.17:0/229217832 shutdown_connections 2024-03-24T03:00:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.836+0000 7fdfcff87640 1 -- 172.21.15.17:0/229217832 wait complete. 2024-03-24T03:00:46.848 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T03:00:46.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.953+0000 7fc8723a9640 1 -- 172.21.15.17:0/2880213930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86c102170 msgr2=0x7fc86c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.953+0000 7fc8723a9640 1 --2- 172.21.15.17:0/2880213930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86c102170 0x7fc86c108a00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc864009ca0 tx=0x7fc86402c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.953+0000 7fc8723a9640 1 -- 172.21.15.17:0/2880213930 shutdown_connections 2024-03-24T03:00:46.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.953+0000 7fc8723a9640 1 --2- 172.21.15.17:0/2880213930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86c102170 0x7fc86c108a00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.953+0000 7fc8723a9640 1 --2- 172.21.15.17:0/2880213930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc86c1017b0 0x7fc86c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.953+0000 7fc8723a9640 1 --2- 172.21.15.17:0/2880213930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c1005c0 0x7fc86c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.953+0000 7fc8723a9640 1 -- 172.21.15.17:0/2880213930 >> 172.21.15.17:0/2880213930 conn(0x7fc86c0fbb30 msgr2=0x7fc86c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.953+0000 7fc8723a9640 1 -- 172.21.15.17:0/2880213930 shutdown_connections 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.953+0000 7fc8723a9640 1 -- 172.21.15.17:0/2880213930 wait complete. 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.953+0000 7fc8723a9640 1 Processor -- start 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.953+0000 7fc8723a9640 1 -- start start 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc8723a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc86c1005c0 0x7fc86c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc8723a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c1017b0 0x7fc86c196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc8723a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86c102170 0x7fc86c19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc8723a9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c10c240 con 0x7fc86c102170 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc8723a9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c10c0c0 con 0x7fc86c1005c0 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc8723a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c10c3c0 con 0x7fc86c1017b0 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc870946640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86c102170 0x7fc86c19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc86bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc86c1005c0 0x7fc86c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc86bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc86c1005c0 0x7fc86c1968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:41070/0 (socket says 172.21.15.17:41070) 2024-03-24T03:00:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc870946640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86c102170 0x7fc86c19de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41734/0 (socket says 172.21.15.17:41734) 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc86bfff640 1 -- 172.21.15.17:0/3223392321 learned_addr learned my addr 172.21.15.17:0/3223392321 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc86b7fe640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c1017b0 0x7fc86c196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc86bfff640 1 -- 172.21.15.17:0/3223392321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c1017b0 msgr2=0x7fc86c196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc86bfff640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c1017b0 0x7fc86c196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc86bfff640 1 -- 172.21.15.17:0/3223392321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86c102170 msgr2=0x7fc86c19de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc86bfff640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86c102170 0x7fc86c19de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc86bfff640 1 -- 172.21.15.17:0/3223392321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c19e4d0 con 0x7fc86c1005c0 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc870946640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86c102170 0x7fc86c19de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.954+0000 7fc86bfff640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc86c1005c0 0x7fc86c1968b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e9c0 tx=0x7fc85c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.955+0000 7fc8697fa640 1 -- 172.21.15.17:0/3223392321 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00cd80 con 0x7fc86c1005c0 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.955+0000 7fc8697fa640 1 -- 172.21.15.17:0/3223392321 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc85c0035f0 con 0x7fc86c1005c0 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.955+0000 7fc8723a9640 1 -- 172.21.15.17:0/3223392321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c068860 con 0x7fc86c1005c0 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.955+0000 7fc8697fa640 1 -- 172.21.15.17:0/3223392321 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c010690 con 0x7fc86c1005c0 2024-03-24T03:00:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.955+0000 7fc8723a9640 1 -- 172.21.15.17:0/3223392321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c068d60 con 0x7fc86c1005c0 2024-03-24T03:00:46.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.956+0000 7fc8723a9640 1 -- 172.21.15.17:0/3223392321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c068490 con 0x7fc86c1005c0 2024-03-24T03:00:46.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.957+0000 7fc8697fa640 1 -- 172.21.15.17:0/3223392321 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc85c0037a0 con 0x7fc86c1005c0 2024-03-24T03:00:46.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.957+0000 7fc8697fa640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8540a2800 0x7fc8540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.957+0000 7fc86b7fe640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8540a2800 0x7fc8540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.957+0000 7fc8697fa640 1 -- 172.21.15.17:0/3223392321 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fc85c0cf200 con 0x7fc86c1005c0 2024-03-24T03:00:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.958+0000 7fc86b7fe640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8540a2800 0x7fc8540a4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc86c197dc0 tx=0x7fc860010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:46.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:46.962+0000 7fc8697fa640 1 -- 172.21.15.17:0/3223392321 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc85c09be60 con 0x7fc86c1005c0 2024-03-24T03:00:47.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.185+0000 7fc8723a9640 1 -- 172.21.15.17:0/3223392321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc86c107350 con 0x7fc86c1005c0 2024-03-24T03:00:47.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.186+0000 7fc8697fa640 1 -- 172.21.15.17:0/3223392321 <== mon.2 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11452 (secure 0 0 0) 0x7fc86c107350 con 0x7fc86c1005c0 2024-03-24T03:00:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:47.187 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T02:58:36.476051+0000","modified":"2024-03-24T02:58:59.074424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24529,"mds_1":24521,"mds_2":24511,"mds_3":24493,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6841/1592961709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":1592961709},{"type":"v1","addr":"172.21.15.17:6841","nonce":1592961709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6839/2102503456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2102503456},{"type":"v1","addr":"172.21.15.19:6839","nonce":2102503456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24511,24529,24485,24493,24521]},"id":1}]} 2024-03-24T03:00:47.187 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 28 2024-03-24T03:00:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.188+0000 7fc8723a9640 1 -- 172.21.15.17:0/3223392321 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8540a2800 msgr2=0x7fc8540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.188+0000 7fc8723a9640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8540a2800 0x7fc8540a4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc86c197dc0 tx=0x7fc860010040 comp rx=0 tx=0).stop 2024-03-24T03:00:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.188+0000 7fc8723a9640 1 -- 172.21.15.17:0/3223392321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc86c1005c0 msgr2=0x7fc86c1968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.188+0000 7fc8723a9640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc86c1005c0 0x7fc86c1968b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e9c0 tx=0x7fc85c00ee80 comp rx=0 tx=0).stop 2024-03-24T03:00:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.188+0000 7fc8723a9640 1 -- 172.21.15.17:0/3223392321 shutdown_connections 2024-03-24T03:00:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.188+0000 7fc8723a9640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc8540a2800 0x7fc8540a4cc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.188+0000 7fc8723a9640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc86c102170 0x7fc86c19de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.188+0000 7fc8723a9640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c1017b0 0x7fc86c196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.188+0000 7fc8723a9640 1 --2- 172.21.15.17:0/3223392321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc86c1005c0 0x7fc86c1968b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.188+0000 7fc8723a9640 1 -- 172.21.15.17:0/3223392321 >> 172.21.15.17:0/3223392321 conn(0x7fc86c0fbb30 msgr2=0x7fc86c104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.188+0000 7fc8723a9640 1 -- 172.21.15.17:0/3223392321 shutdown_connections 2024-03-24T03:00:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.188+0000 7fc8723a9640 1 -- 172.21.15.17:0/3223392321 wait complete. 2024-03-24T03:00:47.199 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:46 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2639292344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:00:47.199 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:46 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/544048140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:00:47.199 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:46 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1927939564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:00:47.199 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:46 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/229217832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:00:47.200 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T03:00:47.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:46 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2639292344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:00:47.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:46 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/544048140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:00:47.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:46 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1927939564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:00:47.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:46 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/229217832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:00:47.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.302+0000 7fe0a7e75640 1 -- 172.21.15.17:0/273085817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a01017d0 msgr2=0x7fe0a0101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.302+0000 7fe0a7e75640 1 --2- 172.21.15.17:0/273085817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a01017d0 0x7fe0a0101c50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe090009900 tx=0x7fe09002bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:47.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.302+0000 7fe0a7e75640 1 -- 172.21.15.17:0/273085817 shutdown_connections 2024-03-24T03:00:47.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.302+0000 7fe0a7e75640 1 --2- 172.21.15.17:0/273085817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0a0102190 0x7fe0a0108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.302+0000 7fe0a7e75640 1 --2- 172.21.15.17:0/273085817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a01017d0 0x7fe0a0101c50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.302+0000 7fe0a7e75640 1 --2- 172.21.15.17:0/273085817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0a01005e0 0x7fe0a0100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.302+0000 7fe0a7e75640 1 -- 172.21.15.17:0/273085817 >> 172.21.15.17:0/273085817 conn(0x7fe0a00fbb50 msgr2=0x7fe0a00fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:47.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.302+0000 7fe0a7e75640 1 -- 172.21.15.17:0/273085817 shutdown_connections 2024-03-24T03:00:47.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.302+0000 7fe0a7e75640 1 -- 172.21.15.17:0/273085817 wait complete. 2024-03-24T03:00:47.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a7e75640 1 Processor -- start 2024-03-24T03:00:47.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a7e75640 1 -- start start 2024-03-24T03:00:47.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a7e75640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0a01005e0 0x7fe0a006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:47.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a7e75640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0a01017d0 0x7fe0a019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:47.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a7e75640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a0102190 0x7fe0a019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a7e75640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a010c260 con 0x7fe0a0102190 2024-03-24T03:00:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a7e75640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a010c0e0 con 0x7fe0a01017d0 2024-03-24T03:00:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a7e75640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a010c3e0 con 0x7fe0a01005e0 2024-03-24T03:00:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a6412640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a0102190 0x7fe0a019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a5c11640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0a01005e0 0x7fe0a006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a5410640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0a01017d0 0x7fe0a019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a6412640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a0102190 0x7fe0a019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41754/0 (socket says 172.21.15.17:41754) 2024-03-24T03:00:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a5c11640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0a01005e0 0x7fe0a006a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:42770/0 (socket says 172.21.15.17:42770) 2024-03-24T03:00:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a6412640 1 -- 172.21.15.17:0/2593187340 learned_addr learned my addr 172.21.15.17:0/2593187340 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a6412640 1 -- 172.21.15.17:0/2593187340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0a01005e0 msgr2=0x7fe0a006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a6412640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0a01005e0 0x7fe0a006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a6412640 1 -- 172.21.15.17:0/2593187340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0a01017d0 msgr2=0x7fe0a019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a6412640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0a01017d0 0x7fe0a019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a6412640 1 -- 172.21.15.17:0/2593187340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a019f5e0 con 0x7fe0a0102190 2024-03-24T03:00:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a5410640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0a01017d0 0x7fe0a019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.303+0000 7fe0a6412640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a0102190 0x7fe0a019ed20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe09800eb10 tx=0x7fe09800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.304+0000 7fe096ffd640 1 -- 172.21.15.17:0/2593187340 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098004b20 con 0x7fe0a0102190 2024-03-24T03:00:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.304+0000 7fe096ffd640 1 -- 172.21.15.17:0/2593187340 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe098004cc0 con 0x7fe0a0102190 2024-03-24T03:00:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.304+0000 7fe0a7e75640 1 -- 172.21.15.17:0/2593187340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a019f8d0 con 0x7fe0a0102190 2024-03-24T03:00:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.304+0000 7fe096ffd640 1 -- 172.21.15.17:0/2593187340 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098010760 con 0x7fe0a0102190 2024-03-24T03:00:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.304+0000 7fe0a7e75640 1 -- 172.21.15.17:0/2593187340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a019fdf0 con 0x7fe0a0102190 2024-03-24T03:00:47.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.305+0000 7fe0a7e75640 1 -- 172.21.15.17:0/2593187340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a01078a0 con 0x7fe0a0102190 2024-03-24T03:00:47.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.306+0000 7fe096ffd640 1 -- 172.21.15.17:0/2593187340 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe098005020 con 0x7fe0a0102190 2024-03-24T03:00:47.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.306+0000 7fe096ffd640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe08c0a2800 0x7fe08c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:47.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.306+0000 7fe096ffd640 1 -- 172.21.15.17:0/2593187340 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fe0980cf580 con 0x7fe0a0102190 2024-03-24T03:00:47.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.306+0000 7fe0a5c11640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe08c0a2800 0x7fe08c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:47.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.307+0000 7fe0a5c11640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe08c0a2800 0x7fe08c0a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0101630 tx=0x7fe09c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.311+0000 7fe096ffd640 1 -- 172.21.15.17:0/2593187340 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe09809c1e0 con 0x7fe0a0102190 2024-03-24T03:00:47.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:46 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2639292344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:00:47.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:46 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/544048140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:00:47.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:46 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1927939564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:00:47.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:46 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/229217832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:00:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.497+0000 7fe0a7e75640 1 -- 172.21.15.17:0/2593187340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0a002d070 con 0x7fe0a0102190 2024-03-24T03:00:47.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.499+0000 7fe096ffd640 1 -- 172.21.15.17:0/2593187340 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24955 (secure 0 0 0) 0x7fe09809bae0 con 0x7fe0a0102190 2024-03-24T03:00:47.500 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:47.500 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":93,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T03:00:31.817661+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T03:00:47.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.501+0000 7fe0a7e75640 1 -- 172.21.15.17:0/2593187340 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe08c0a2800 msgr2=0x7fe08c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.501+0000 7fe0a7e75640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe08c0a2800 0x7fe08c0a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0101630 tx=0x7fe09c005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:47.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.501+0000 7fe0a7e75640 1 -- 172.21.15.17:0/2593187340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a0102190 msgr2=0x7fe0a019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.501+0000 7fe0a7e75640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a0102190 0x7fe0a019ed20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe09800eb10 tx=0x7fe09800c2d0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.501+0000 7fe0a7e75640 1 -- 172.21.15.17:0/2593187340 shutdown_connections 2024-03-24T03:00:47.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.501+0000 7fe0a7e75640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe08c0a2800 0x7fe08c0a4cc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.501+0000 7fe0a7e75640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a0102190 0x7fe0a019ed20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.501+0000 7fe0a7e75640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0a01017d0 0x7fe0a019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.501+0000 7fe0a7e75640 1 --2- 172.21.15.17:0/2593187340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0a01005e0 0x7fe0a006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.501+0000 7fe0a7e75640 1 -- 172.21.15.17:0/2593187340 >> 172.21.15.17:0/2593187340 conn(0x7fe0a00fbb50 msgr2=0x7fe0a0104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:47.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.501+0000 7fe0a7e75640 1 -- 172.21.15.17:0/2593187340 shutdown_connections 2024-03-24T03:00:47.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.501+0000 7fe0a7e75640 1 -- 172.21.15.17:0/2593187340 wait complete. 2024-03-24T03:00:47.513 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T02:53:54.207139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-24T02:58:34.785192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-24T02:58:35.679772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-24T02:58:39.221772+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-24T03:00:47.514 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T03:00:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.619+0000 7f33507ab640 1 -- 172.21.15.17:0/974449374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3348102160 msgr2=0x7f33481089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.619+0000 7f33507ab640 1 --2- 172.21.15.17:0/974449374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3348102160 0x7f33481089f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3338009be0 tx=0x7f333802bef0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 -- 172.21.15.17:0/974449374 shutdown_connections 2024-03-24T03:00:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 --2- 172.21.15.17:0/974449374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3348102160 0x7f33481089f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 --2- 172.21.15.17:0/974449374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33481017a0 0x7f3348101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 --2- 172.21.15.17:0/974449374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33481005b0 0x7f33481009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 -- 172.21.15.17:0/974449374 >> 172.21.15.17:0/974449374 conn(0x7f33480fbb40 msgr2=0x7f33480fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 -- 172.21.15.17:0/974449374 shutdown_connections 2024-03-24T03:00:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 -- 172.21.15.17:0/974449374 wait complete. 2024-03-24T03:00:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 Processor -- start 2024-03-24T03:00:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 -- start start 2024-03-24T03:00:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33481005b0 0x7f334810adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33481017a0 0x7f33481093f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3348102160 0x7f3348109930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334810d350 con 0x7f33481017a0 2024-03-24T03:00:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334810d1d0 con 0x7f3348102160 2024-03-24T03:00:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f33507ab640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334810d4d0 con 0x7f33481005b0 2024-03-24T03:00:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.620+0000 7f334dd46640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33481017a0 0x7f33481093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f334dd46640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33481017a0 0x7f33481093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41772/0 (socket says 172.21.15.17:41772) 2024-03-24T03:00:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f334dd46640 1 -- 172.21.15.17:0/4160110632 learned_addr learned my addr 172.21.15.17:0/4160110632 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f334e547640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33481005b0 0x7f334810adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f334ed48640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3348102160 0x7f3348109930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f334dd46640 1 -- 172.21.15.17:0/4160110632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33481005b0 msgr2=0x7f334810adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f334dd46640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33481005b0 0x7f334810adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f334dd46640 1 -- 172.21.15.17:0/4160110632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3348102160 msgr2=0x7f3348109930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f334dd46640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3348102160 0x7f3348109930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f334dd46640 1 -- 172.21.15.17:0/4160110632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334810a1c0 con 0x7f33481017a0 2024-03-24T03:00:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f334e547640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33481005b0 0x7f334810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f334dd46640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33481017a0 0x7f33481093f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f333c00e9c0 tx=0x7f333c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f33477fe640 1 -- 172.21.15.17:0/4160110632 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c00ccc0 con 0x7f33481017a0 2024-03-24T03:00:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f33507ab640 1 -- 172.21.15.17:0/4160110632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33481a1ad0 con 0x7f33481017a0 2024-03-24T03:00:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f33477fe640 1 -- 172.21.15.17:0/4160110632 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f333c005500 con 0x7f33481017a0 2024-03-24T03:00:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f33477fe640 1 -- 172.21.15.17:0/4160110632 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c010430 con 0x7f33481017a0 2024-03-24T03:00:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.621+0000 7f33507ab640 1 -- 172.21.15.17:0/4160110632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33481a2070 con 0x7f33481017a0 2024-03-24T03:00:47.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.622+0000 7f33507ab640 1 -- 172.21.15.17:0/4160110632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33481078d0 con 0x7f33481017a0 2024-03-24T03:00:47.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.623+0000 7f33477fe640 1 -- 172.21.15.17:0/4160110632 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f333c0046c0 con 0x7f33481017a0 2024-03-24T03:00:47.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.624+0000 7f33477fe640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f33300a2800 0x7f33300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:47.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.624+0000 7f334e547640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f33300a2800 0x7f33300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:47.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.624+0000 7f33477fe640 1 -- 172.21.15.17:0/4160110632 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f333c0cf400 con 0x7f33481017a0 2024-03-24T03:00:47.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.624+0000 7f334e547640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f33300a2800 0x7f33300a4cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3348101600 tx=0x7f3340005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:47.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.627+0000 7f33477fe640 1 -- 172.21.15.17:0/4160110632 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f333c053740 con 0x7f33481017a0 2024-03-24T03:00:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.808+0000 7f33507ab640 1 -- 172.21.15.17:0/4160110632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f334802d070 con 0x7f33481017a0 2024-03-24T03:00:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.809+0000 7f33477fe640 1 -- 172.21.15.17:0/4160110632 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f333c09c000 con 0x7f33481017a0 2024-03-24T03:00:47.809 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-24T03:00:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.811+0000 7f33507ab640 1 -- 172.21.15.17:0/4160110632 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f33300a2800 msgr2=0x7f33300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.811+0000 7f33507ab640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f33300a2800 0x7f33300a4cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3348101600 tx=0x7f3340005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.811+0000 7f33507ab640 1 -- 172.21.15.17:0/4160110632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33481017a0 msgr2=0x7f33481093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.811+0000 7f33507ab640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33481017a0 0x7f33481093f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f333c00e9c0 tx=0x7f333c00ee80 comp rx=0 tx=0).stop 2024-03-24T03:00:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.811+0000 7f33507ab640 1 -- 172.21.15.17:0/4160110632 shutdown_connections 2024-03-24T03:00:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.811+0000 7f33507ab640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f33300a2800 0x7f33300a4cc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.811+0000 7f33507ab640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3348102160 0x7f3348109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.811+0000 7f33507ab640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33481017a0 0x7f33481093f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.811+0000 7f33507ab640 1 --2- 172.21.15.17:0/4160110632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33481005b0 0x7f334810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.811+0000 7f33507ab640 1 -- 172.21.15.17:0/4160110632 >> 172.21.15.17:0/4160110632 conn(0x7f33480fbb40 msgr2=0x7f3348104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.811+0000 7f33507ab640 1 -- 172.21.15.17:0/4160110632 shutdown_connections 2024-03-24T03:00:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.811+0000 7f33507ab640 1 -- 172.21.15.17:0/4160110632 wait complete. 2024-03-24T03:00:47.822 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-24T03:00:47.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 -- 172.21.15.17:0/2561830690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd41016d0 msgr2=0x7f4fd4101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 --2- 172.21.15.17:0/2561830690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd41016d0 0x7f4fd4101af0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009900 tx=0x7f4fcc02bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:47.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 -- 172.21.15.17:0/2561830690 shutdown_connections 2024-03-24T03:00:47.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 --2- 172.21.15.17:0/2561830690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd4103280 0x7f4fd4109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 --2- 172.21.15.17:0/2561830690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fd41028c0 0x7f4fd4102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 --2- 172.21.15.17:0/2561830690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd41016d0 0x7f4fd4101af0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 -- 172.21.15.17:0/2561830690 >> 172.21.15.17:0/2561830690 conn(0x7f4fd40fcc40 msgr2=0x7f4fd40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:47.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 -- 172.21.15.17:0/2561830690 shutdown_connections 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 -- 172.21.15.17:0/2561830690 wait complete. 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 Processor -- start 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 -- start start 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd41016d0 0x7f4fd4197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fd41028c0 0x7f4fd4197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd4103280 0x7f4fd419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd410d390 con 0x7f4fd4103280 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd410d210 con 0x7f4fd41016d0 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.928+0000 7f4fdc20b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd410d510 con 0x7f4fd41028c0 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fda7a8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd4103280 0x7f4fd419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fd9fa7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd41016d0 0x7f4fd4197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fda7a8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd4103280 0x7f4fd419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41796/0 (socket says 172.21.15.17:41796) 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fda7a8640 1 -- 172.21.15.17:0/1272372154 learned_addr learned my addr 172.21.15.17:0/1272372154 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fd97a6640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fd41028c0 0x7f4fd4197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fda7a8640 1 -- 172.21.15.17:0/1272372154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fd41028c0 msgr2=0x7f4fd4197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fda7a8640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fd41028c0 0x7f4fd4197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fda7a8640 1 -- 172.21.15.17:0/1272372154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd41016d0 msgr2=0x7f4fd4197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fda7a8640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd41016d0 0x7f4fd4197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fda7a8640 1 -- 172.21.15.17:0/1272372154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd419f620 con 0x7f4fd4103280 2024-03-24T03:00:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fd9fa7640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd41016d0 0x7f4fd4197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fda7a8640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd4103280 0x7f4fd419efb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400eb10 tx=0x7f4fc400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fd2ffd640 1 -- 172.21.15.17:0/1272372154 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4004b20 con 0x7f4fd4103280 2024-03-24T03:00:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fd2ffd640 1 -- 172.21.15.17:0/1272372154 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fc4004cc0 con 0x7f4fd4103280 2024-03-24T03:00:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fdc20b640 1 -- 172.21.15.17:0/1272372154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd419f910 con 0x7f4fd4103280 2024-03-24T03:00:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fd2ffd640 1 -- 172.21.15.17:0/1272372154 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4010760 con 0x7f4fd4103280 2024-03-24T03:00:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.929+0000 7f4fdc20b640 1 -- 172.21.15.17:0/1272372154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd419fe10 con 0x7f4fd4103280 2024-03-24T03:00:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.930+0000 7f4fdc20b640 1 -- 172.21.15.17:0/1272372154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd41089f0 con 0x7f4fd4103280 2024-03-24T03:00:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.932+0000 7f4fd2ffd640 1 -- 172.21.15.17:0/1272372154 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4fc4005000 con 0x7f4fd4103280 2024-03-24T03:00:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.932+0000 7f4fd2ffd640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4fbc0a2800 0x7f4fbc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.933+0000 7f4fd9fa7640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4fbc0a2800 0x7f4fbc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.933+0000 7f4fd2ffd640 1 -- 172.21.15.17:0/1272372154 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f4fc40cf6a0 con 0x7f4fd4103280 2024-03-24T03:00:47.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.933+0000 7f4fd9fa7640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4fbc0a2800 0x7f4fbc0a4cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc02c170 tx=0x7f4fcc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:47.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:47.936+0000 7f4fd2ffd640 1 -- 172.21.15.17:0/1272372154 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4fc4053740 con 0x7f4fd4103280 2024-03-24T03:00:48.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.118+0000 7f4fdc20b640 1 -- 172.21.15.17:0/1272372154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4fd402d2c0 con 0x7f4fd4103280 2024-03-24T03:00:48.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.119+0000 7f4fd2ffd640 1 -- 172.21.15.17:0/1272372154 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4fc409c2a0 con 0x7f4fd4103280 2024-03-24T03:00:48.119 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T03:00:48.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.120+0000 7f4fdc20b640 1 -- 172.21.15.17:0/1272372154 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4fbc0a2800 msgr2=0x7f4fbc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.120+0000 7f4fdc20b640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4fbc0a2800 0x7f4fbc0a4cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc02c170 tx=0x7f4fcc02df90 comp rx=0 tx=0).stop 2024-03-24T03:00:48.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.120+0000 7f4fdc20b640 1 -- 172.21.15.17:0/1272372154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd4103280 msgr2=0x7f4fd419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.120+0000 7f4fdc20b640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd4103280 0x7f4fd419efb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400eb10 tx=0x7f4fc400c2d0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.121+0000 7f4fdc20b640 1 -- 172.21.15.17:0/1272372154 shutdown_connections 2024-03-24T03:00:48.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.121+0000 7f4fdc20b640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f4fbc0a2800 0x7f4fbc0a4cc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.121+0000 7f4fdc20b640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd4103280 0x7f4fd419efb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.121+0000 7f4fdc20b640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fd41028c0 0x7f4fd4197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.121+0000 7f4fdc20b640 1 --2- 172.21.15.17:0/1272372154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd41016d0 0x7f4fd4197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.121+0000 7f4fdc20b640 1 -- 172.21.15.17:0/1272372154 >> 172.21.15.17:0/1272372154 conn(0x7f4fd40fcc40 msgr2=0x7f4fd4105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:48.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.121+0000 7f4fdc20b640 1 -- 172.21.15.17:0/1272372154 shutdown_connections 2024-03-24T03:00:48.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.121+0000 7f4fdc20b640 1 -- 172.21.15.17:0/1272372154 wait complete. 2024-03-24T03:00:48.131 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:47 smithi017 ceph-mon[31926]: pgmap v361: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 104 B/s, 0 objects/s recovering 2024-03-24T03:00:48.131 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:47 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3223392321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:48.131 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:47 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2593187340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:48.131 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:47 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4160110632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:00:48.132 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-24T03:00:48.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.237+0000 7f462a5f4640 1 -- 172.21.15.17:0/3255126036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46241005b0 msgr2=0x7f46241009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.237+0000 7f462a5f4640 1 --2- 172.21.15.17:0/3255126036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46241005b0 0x7f46241009d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4614009900 tx=0x7f461402bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.237+0000 7f462a5f4640 1 -- 172.21.15.17:0/3255126036 shutdown_connections 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.237+0000 7f462a5f4640 1 --2- 172.21.15.17:0/3255126036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4624102160 0x7f46241089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.237+0000 7f462a5f4640 1 --2- 172.21.15.17:0/3255126036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241017a0 0x7f4624101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.237+0000 7f462a5f4640 1 --2- 172.21.15.17:0/3255126036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46241005b0 0x7f46241009d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.237+0000 7f462a5f4640 1 -- 172.21.15.17:0/3255126036 >> 172.21.15.17:0/3255126036 conn(0x7f46240fbb40 msgr2=0x7f46240fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.238+0000 7f462a5f4640 1 -- 172.21.15.17:0/3255126036 shutdown_connections 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.238+0000 7f462a5f4640 1 -- 172.21.15.17:0/3255126036 wait complete. 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.238+0000 7f462a5f4640 1 Processor -- start 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.238+0000 7f462a5f4640 1 -- start start 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.238+0000 7f462a5f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46241005b0 0x7f46241968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.238+0000 7f462a5f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241017a0 0x7f4624196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.238+0000 7f462a5f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4624102160 0x7f462419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.238+0000 7f462a5f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462410c220 con 0x7f4624102160 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.238+0000 7f462a5f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462410c0a0 con 0x7f46241017a0 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.238+0000 7f462a5f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462410c3a0 con 0x7f46241005b0 2024-03-24T03:00:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f4628b91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4624102160 0x7f462419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f46237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241017a0 0x7f4624196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f4628b91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4624102160 0x7f462419de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41818/0 (socket says 172.21.15.17:41818) 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f4628b91640 1 -- 172.21.15.17:0/4092164923 learned_addr learned my addr 172.21.15.17:0/4092164923 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f4623fff640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46241005b0 0x7f46241968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f46237fe640 1 -- 172.21.15.17:0/4092164923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46241005b0 msgr2=0x7f46241968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f46237fe640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46241005b0 0x7f46241968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f46237fe640 1 -- 172.21.15.17:0/4092164923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4624102160 msgr2=0x7f462419de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f46237fe640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4624102160 0x7f462419de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f46237fe640 1 -- 172.21.15.17:0/4092164923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462419e4c0 con 0x7f46241017a0 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f4628b91640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4624102160 0x7f462419de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f46237fe640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241017a0 0x7f4624196de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f461800eb00 tx=0x7f4618009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f46217fa640 1 -- 172.21.15.17:0/4092164923 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461800bc10 con 0x7f46241017a0 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f462a5f4640 1 -- 172.21.15.17:0/4092164923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4624069cd0 con 0x7f46241017a0 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f46217fa640 1 -- 172.21.15.17:0/4092164923 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f461800bdb0 con 0x7f46241017a0 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f46217fa640 1 -- 172.21.15.17:0/4092164923 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618017410 con 0x7f46241017a0 2024-03-24T03:00:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.239+0000 7f462a5f4640 1 -- 172.21.15.17:0/4092164923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462406a270 con 0x7f46241017a0 2024-03-24T03:00:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.240+0000 7f462a5f4640 1 -- 172.21.15.17:0/4092164923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46240688e0 con 0x7f46241017a0 2024-03-24T03:00:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.241+0000 7f46217fa640 1 -- 172.21.15.17:0/4092164923 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4618046a60 con 0x7f46241017a0 2024-03-24T03:00:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.242+0000 7f46217fa640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f460c0a2800 0x7f460c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.242+0000 7f4623fff640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f460c0a2800 0x7f460c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.242+0000 7f46217fa640 1 -- 172.21.15.17:0/4092164923 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f46180d04e0 con 0x7f46241017a0 2024-03-24T03:00:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.242+0000 7f4623fff640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f460c0a2800 0x7f460c0a4cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4614000c00 tx=0x7f4614005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:48.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.245+0000 7f46217fa640 1 -- 172.21.15.17:0/4092164923 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f461809ca90 con 0x7f46241017a0 2024-03-24T03:00:48.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:47 smithi188 ceph-mon[36206]: pgmap v361: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 104 B/s, 0 objects/s recovering 2024-03-24T03:00:48.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:47 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3223392321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:48.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:47 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2593187340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:48.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:47 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4160110632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:00:48.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:47 smithi019 ceph-mon[35944]: pgmap v361: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 104 B/s, 0 objects/s recovering 2024-03-24T03:00:48.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:47 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3223392321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:48.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:47 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2593187340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:48.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:47 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4160110632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:00:48.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.429+0000 7f462a5f4640 1 -- 172.21.15.17:0/4092164923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4624107670 con 0x7f46241017a0 2024-03-24T03:00:48.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.429+0000 7f46217fa640 1 -- 172.21.15.17:0/4092164923 <== mon.2 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46180a0b30 con 0x7f46241017a0 2024-03-24T03:00:48.430 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-24T03:00:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.431+0000 7f462a5f4640 1 -- 172.21.15.17:0/4092164923 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f460c0a2800 msgr2=0x7f460c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.431+0000 7f462a5f4640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f460c0a2800 0x7f460c0a4cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4614000c00 tx=0x7f4614005fd0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.431+0000 7f462a5f4640 1 -- 172.21.15.17:0/4092164923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241017a0 msgr2=0x7f4624196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.431+0000 7f462a5f4640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241017a0 0x7f4624196de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f461800eb00 tx=0x7f4618009f10 comp rx=0 tx=0).stop 2024-03-24T03:00:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.431+0000 7f462a5f4640 1 -- 172.21.15.17:0/4092164923 shutdown_connections 2024-03-24T03:00:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.431+0000 7f462a5f4640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f460c0a2800 0x7f460c0a4cc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.431+0000 7f462a5f4640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4624102160 0x7f462419de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.431+0000 7f462a5f4640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241017a0 0x7f4624196de0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.431+0000 7f462a5f4640 1 --2- 172.21.15.17:0/4092164923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46241005b0 0x7f46241968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.431+0000 7f462a5f4640 1 -- 172.21.15.17:0/4092164923 >> 172.21.15.17:0/4092164923 conn(0x7f46240fbb40 msgr2=0x7f4624104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.431+0000 7f462a5f4640 1 -- 172.21.15.17:0/4092164923 shutdown_connections 2024-03-24T03:00:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.431+0000 7f462a5f4640 1 -- 172.21.15.17:0/4092164923 wait complete. 2024-03-24T03:00:48.443 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-24T03:00:48.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.548+0000 7fabf4cae640 1 -- 172.21.15.17:0/1419155446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf0102150 msgr2=0x7fabf01089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 --2- 172.21.15.17:0/1419155446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf0102150 0x7fabf01089e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009ca0 tx=0x7fabe802c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 -- 172.21.15.17:0/1419155446 shutdown_connections 2024-03-24T03:00:48.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 --2- 172.21.15.17:0/1419155446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf0102150 0x7fabf01089e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 --2- 172.21.15.17:0/1419155446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf0101790 0x7fabf0101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 --2- 172.21.15.17:0/1419155446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf01005a0 0x7fabf01009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 -- 172.21.15.17:0/1419155446 >> 172.21.15.17:0/1419155446 conn(0x7fabf00fbb30 msgr2=0x7fabf00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:48.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 -- 172.21.15.17:0/1419155446 shutdown_connections 2024-03-24T03:00:48.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 -- 172.21.15.17:0/1419155446 wait complete. 2024-03-24T03:00:48.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 Processor -- start 2024-03-24T03:00:48.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 -- start start 2024-03-24T03:00:48.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf01005a0 0x7fabf01979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:48.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf0101790 0x7fabf0197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:48.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf0102150 0x7fabf019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:48.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf010c230 con 0x7fabf01005a0 2024-03-24T03:00:48.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf010c0b0 con 0x7fabf0101790 2024-03-24T03:00:48.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.549+0000 7fabf4cae640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf010c3b0 con 0x7fabf0102150 2024-03-24T03:00:48.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabee59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf01005a0 0x7fabf01979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabee59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf01005a0 0x7fabf01979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41836/0 (socket says 172.21.15.17:41836) 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabee59c640 1 -- 172.21.15.17:0/518916820 learned_addr learned my addr 172.21.15.17:0/518916820 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabe7fff640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf0101790 0x7fabf0197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabeed9d640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf0102150 0x7fabf019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabee59c640 1 -- 172.21.15.17:0/518916820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf0102150 msgr2=0x7fabf019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabee59c640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf0102150 0x7fabf019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabee59c640 1 -- 172.21.15.17:0/518916820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf0101790 msgr2=0x7fabf0197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabee59c640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf0101790 0x7fabf0197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabee59c640 1 -- 172.21.15.17:0/518916820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf019f5c0 con 0x7fabf01005a0 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabe7fff640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf0101790 0x7fabf0197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabee59c640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf01005a0 0x7fabf01979a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fabe000e9c0 tx=0x7fabe000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabe77fe640 1 -- 172.21.15.17:0/518916820 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe000cd80 con 0x7fabf01005a0 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabe77fe640 1 -- 172.21.15.17:0/518916820 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabe00035f0 con 0x7fabf01005a0 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabf4cae640 1 -- 172.21.15.17:0/518916820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf019f8b0 con 0x7fabf01005a0 2024-03-24T03:00:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabe77fe640 1 -- 172.21.15.17:0/518916820 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe0010690 con 0x7fabf01005a0 2024-03-24T03:00:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.550+0000 7fabf4cae640 1 -- 172.21.15.17:0/518916820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf019fdb0 con 0x7fabf01005a0 2024-03-24T03:00:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.551+0000 7fabf4cae640 1 -- 172.21.15.17:0/518916820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf01a0010 con 0x7fabf01005a0 2024-03-24T03:00:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.552+0000 7fabe77fe640 1 -- 172.21.15.17:0/518916820 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fabe00037a0 con 0x7fabf01005a0 2024-03-24T03:00:48.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.553+0000 7fabe77fe640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fabdc0a2800 0x7fabdc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:48.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.553+0000 7fabe7fff640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fabdc0a2800 0x7fabdc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:48.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.553+0000 7fabe77fe640 1 -- 172.21.15.17:0/518916820 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fabe00cf260 con 0x7fabf01005a0 2024-03-24T03:00:48.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.553+0000 7fabe7fff640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fabdc0a2800 0x7fabdc0a4cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fabd80095f0 tx=0x7fabd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.557+0000 7fabe77fe640 1 -- 172.21.15.17:0/518916820 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fabe009be60 con 0x7fabf01005a0 2024-03-24T03:00:48.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.741+0000 7fabf4cae640 1 -- 172.21.15.17:0/518916820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fabf01a01e0 con 0x7fabf01005a0 2024-03-24T03:00:48.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.742+0000 7fabe77fe640 1 -- 172.21.15.17:0/518916820 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fabe009b760 con 0x7fabf01005a0 2024-03-24T03:00:48.742 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T03:00:48.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.743+0000 7fabf4cae640 1 -- 172.21.15.17:0/518916820 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fabdc0a2800 msgr2=0x7fabdc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.743+0000 7fabf4cae640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fabdc0a2800 0x7fabdc0a4cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fabd80095f0 tx=0x7fabd8005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:48.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.743+0000 7fabf4cae640 1 -- 172.21.15.17:0/518916820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf01005a0 msgr2=0x7fabf01979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.743+0000 7fabf4cae640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf01005a0 0x7fabf01979a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fabe000e9c0 tx=0x7fabe000ee80 comp rx=0 tx=0).stop 2024-03-24T03:00:48.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.744+0000 7fabf4cae640 1 -- 172.21.15.17:0/518916820 shutdown_connections 2024-03-24T03:00:48.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.744+0000 7fabf4cae640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fabdc0a2800 0x7fabdc0a4cc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.744+0000 7fabf4cae640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabf0102150 0x7fabf019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.744+0000 7fabf4cae640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf0101790 0x7fabf0197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.744+0000 7fabf4cae640 1 --2- 172.21.15.17:0/518916820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf01005a0 0x7fabf01979a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.744+0000 7fabf4cae640 1 -- 172.21.15.17:0/518916820 >> 172.21.15.17:0/518916820 conn(0x7fabf00fbb30 msgr2=0x7fabf00fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:48.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.744+0000 7fabf4cae640 1 -- 172.21.15.17:0/518916820 shutdown_connections 2024-03-24T03:00:48.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.744+0000 7fabf4cae640 1 -- 172.21.15.17:0/518916820 wait complete. 2024-03-24T03:00:48.755 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T03:00:48.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.858+0000 7ff3100de640 1 -- 172.21.15.17:0/4042307432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff308104a70 msgr2=0x7ff308106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.858+0000 7ff3100de640 1 --2- 172.21.15.17:0/4042307432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff308104a70 0x7ff308106e60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009900 tx=0x7ff2fc02bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:48.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.858+0000 7ff3100de640 1 -- 172.21.15.17:0/4042307432 shutdown_connections 2024-03-24T03:00:48.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.858+0000 7ff3100de640 1 --2- 172.21.15.17:0/4042307432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3081073a0 0x7ff3081097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.858+0000 7ff3100de640 1 --2- 172.21.15.17:0/4042307432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff308104a70 0x7ff308106e60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.858+0000 7ff3100de640 1 --2- 172.21.15.17:0/4042307432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff308102140 0x7ff308104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.858+0000 7ff3100de640 1 -- 172.21.15.17:0/4042307432 >> 172.21.15.17:0/4042307432 conn(0x7ff3080fbb30 msgr2=0x7ff3080fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:48.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.858+0000 7ff3100de640 1 -- 172.21.15.17:0/4042307432 shutdown_connections 2024-03-24T03:00:48.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.858+0000 7ff3100de640 1 -- 172.21.15.17:0/4042307432 wait complete. 2024-03-24T03:00:48.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff3100de640 1 Processor -- start 2024-03-24T03:00:48.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff3100de640 1 -- start start 2024-03-24T03:00:48.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff3100de640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff308102140 0x7ff308197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:48.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff3100de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff308104a70 0x7ff308197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff3100de640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3081073a0 0x7ff30819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff3100de640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30810d340 con 0x7ff308102140 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff3100de640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30810d1c0 con 0x7ff308104a70 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff3100de640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30810d4c0 con 0x7ff3081073a0 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff30de7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff308102140 0x7ff308197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff30de7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff308102140 0x7ff308197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41846/0 (socket says 172.21.15.17:41846) 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff30de7a640 1 -- 172.21.15.17:0/3270391664 learned_addr learned my addr 172.21.15.17:0/3270391664 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff30d679640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff308104a70 0x7ff308197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff30e67b640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3081073a0 0x7ff30819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff30de7a640 1 -- 172.21.15.17:0/3270391664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3081073a0 msgr2=0x7ff30819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff30de7a640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3081073a0 0x7ff30819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff30de7a640 1 -- 172.21.15.17:0/3270391664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff308104a70 msgr2=0x7ff308197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff30de7a640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff308104a70 0x7ff308197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:48.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff30de7a640 1 -- 172.21.15.17:0/3270391664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30819f630 con 0x7ff308102140 2024-03-24T03:00:48.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.859+0000 7ff30e67b640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3081073a0 0x7ff30819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T03:00:48.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.860+0000 7ff30de7a640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff308102140 0x7ff308197a10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff30000ebc0 tx=0x7ff30000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:48.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.860+0000 7ff306ffd640 1 -- 172.21.15.17:0/3270391664 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300010630 con 0x7ff308102140 2024-03-24T03:00:48.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.860+0000 7ff3100de640 1 -- 172.21.15.17:0/3270391664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30819f920 con 0x7ff308102140 2024-03-24T03:00:48.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.860+0000 7ff306ffd640 1 -- 172.21.15.17:0/3270391664 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff300010cb0 con 0x7ff308102140 2024-03-24T03:00:48.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.860+0000 7ff306ffd640 1 -- 172.21.15.17:0/3270391664 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300018bd0 con 0x7ff308102140 2024-03-24T03:00:48.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.860+0000 7ff3100de640 1 -- 172.21.15.17:0/3270391664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30819fe40 con 0x7ff308102140 2024-03-24T03:00:48.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.861+0000 7ff3100de640 1 -- 172.21.15.17:0/3270391664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3080fd620 con 0x7ff308102140 2024-03-24T03:00:48.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.862+0000 7ff306ffd640 1 -- 172.21.15.17:0/3270391664 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff3000107d0 con 0x7ff308102140 2024-03-24T03:00:48.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.862+0000 7ff306ffd640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff2f00a2800 0x7ff2f00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:48.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.863+0000 7ff306ffd640 1 -- 172.21.15.17:0/3270391664 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7ff3000cf530 con 0x7ff308102140 2024-03-24T03:00:48.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.863+0000 7ff30d679640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff2f00a2800 0x7ff2f00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:48.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.863+0000 7ff30d679640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff2f00a2800 0x7ff2f00a4cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc000c00 tx=0x7ff2fc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:48.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:48.867+0000 7ff306ffd640 1 -- 172.21.15.17:0/3270391664 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff30009c190 con 0x7ff308102140 2024-03-24T03:00:49.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.093+0000 7ff3100de640 1 -- 172.21.15.17:0/3270391664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff30802d070 con 0x7ff308102140 2024-03-24T03:00:49.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.094+0000 7ff306ffd640 1 -- 172.21.15.17:0/3270391664 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11452 (secure 0 0 0) 0x7ff30009ba90 con 0x7ff308102140 2024-03-24T03:00:49.094 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:49.094 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T02:58:36.476051+0000","modified":"2024-03-24T02:58:59.074424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24529,"mds_1":24521,"mds_2":24511,"mds_3":24493,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6841/1592961709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":1592961709},{"type":"v1","addr":"172.21.15.17:6841","nonce":1592961709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6839/2102503456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2102503456},{"type":"v1","addr":"172.21.15.19:6839","nonce":2102503456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24511,24529,24485,24493,24521]},"id":1}]} 2024-03-24T03:00:49.095 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 28 2024-03-24T03:00:49.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.096+0000 7ff3100de640 1 -- 172.21.15.17:0/3270391664 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff2f00a2800 msgr2=0x7ff2f00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.096+0000 7ff3100de640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff2f00a2800 0x7ff2f00a4cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc000c00 tx=0x7ff2fc005fd0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.096+0000 7ff3100de640 1 -- 172.21.15.17:0/3270391664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff308102140 msgr2=0x7ff308197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.096+0000 7ff3100de640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff308102140 0x7ff308197a10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff30000ebc0 tx=0x7ff30000c6a0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.096+0000 7ff3100de640 1 -- 172.21.15.17:0/3270391664 shutdown_connections 2024-03-24T03:00:49.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.096+0000 7ff3100de640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7ff2f00a2800 0x7ff2f00a4cc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.096+0000 7ff3100de640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3081073a0 0x7ff30819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.096+0000 7ff3100de640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff308104a70 0x7ff308197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.096+0000 7ff3100de640 1 --2- 172.21.15.17:0/3270391664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff308102140 0x7ff308197a10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.096+0000 7ff3100de640 1 -- 172.21.15.17:0/3270391664 >> 172.21.15.17:0/3270391664 conn(0x7ff3080fbb30 msgr2=0x7ff308102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.096+0000 7ff3100de640 1 -- 172.21.15.17:0/3270391664 shutdown_connections 2024-03-24T03:00:49.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.096+0000 7ff3100de640 1 -- 172.21.15.17:0/3270391664 wait complete. 2024-03-24T03:00:49.107 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:48 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1272372154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:00:49.107 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:48 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4092164923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:00:49.107 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:48 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/518916820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:00:49.108 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T03:00:49.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.211+0000 7fc190081640 1 -- 172.21.15.17:0/2363278857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc188102170 msgr2=0x7fc188108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.211+0000 7fc190081640 1 --2- 172.21.15.17:0/2363278857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc188102170 0x7fc188108a00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc180009ca0 tx=0x7fc18002c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.211+0000 7fc190081640 1 -- 172.21.15.17:0/2363278857 shutdown_connections 2024-03-24T03:00:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.211+0000 7fc190081640 1 --2- 172.21.15.17:0/2363278857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc188102170 0x7fc188108a00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.211+0000 7fc190081640 1 --2- 172.21.15.17:0/2363278857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1881017b0 0x7fc188101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.211+0000 7fc190081640 1 --2- 172.21.15.17:0/2363278857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1881005c0 0x7fc1881009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.211+0000 7fc190081640 1 -- 172.21.15.17:0/2363278857 >> 172.21.15.17:0/2363278857 conn(0x7fc1880fbb30 msgr2=0x7fc1880fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.211+0000 7fc190081640 1 -- 172.21.15.17:0/2363278857 shutdown_connections 2024-03-24T03:00:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.211+0000 7fc190081640 1 -- 172.21.15.17:0/2363278857 wait complete. 2024-03-24T03:00:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.211+0000 7fc190081640 1 Processor -- start 2024-03-24T03:00:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.211+0000 7fc190081640 1 -- start start 2024-03-24T03:00:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc190081640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1881005c0 0x7fc1881979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc190081640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1881017b0 0x7fc188197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc190081640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc188102170 0x7fc18819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc190081640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18810c220 con 0x7fc1881017b0 2024-03-24T03:00:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc190081640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18810c0a0 con 0x7fc188102170 2024-03-24T03:00:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc190081640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18810c3a0 con 0x7fc1881005c0 2024-03-24T03:00:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc18d61c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1881017b0 0x7fc188197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc18d61c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1881017b0 0x7fc188197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41872/0 (socket says 172.21.15.17:41872) 2024-03-24T03:00:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc18d61c640 1 -- 172.21.15.17:0/1865167529 learned_addr learned my addr 172.21.15.17:0/1865167529 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc18de1d640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1881005c0 0x7fc1881979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc18e61e640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc188102170 0x7fc18819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc18d61c640 1 -- 172.21.15.17:0/1865167529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1881005c0 msgr2=0x7fc1881979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc18d61c640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1881005c0 0x7fc1881979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc18d61c640 1 -- 172.21.15.17:0/1865167529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc188102170 msgr2=0x7fc18819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc18d61c640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc188102170 0x7fc18819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc18d61c640 1 -- 172.21.15.17:0/1865167529 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18819f5d0 con 0x7fc1881017b0 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc18d61c640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1881017b0 0x7fc188197ef0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e9f0 tx=0x7fc18400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc17effd640 1 -- 172.21.15.17:0/1865167529 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18400cd80 con 0x7fc1881017b0 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc190081640 1 -- 172.21.15.17:0/1865167529 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18819f8c0 con 0x7fc1881017b0 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc17effd640 1 -- 172.21.15.17:0/1865167529 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc184010910 con 0x7fc1881017b0 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc17effd640 1 -- 172.21.15.17:0/1865167529 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1840188f0 con 0x7fc1881017b0 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.212+0000 7fc190081640 1 -- 172.21.15.17:0/1865167529 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18819fe60 con 0x7fc1881017b0 2024-03-24T03:00:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.214+0000 7fc190081640 1 -- 172.21.15.17:0/1865167529 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1881a00c0 con 0x7fc1881017b0 2024-03-24T03:00:49.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.214+0000 7fc17effd640 1 -- 172.21.15.17:0/1865167529 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc184010ac0 con 0x7fc1881017b0 2024-03-24T03:00:49.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.215+0000 7fc17effd640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc1740a2800 0x7fc1740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.215+0000 7fc18de1d640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc1740a2800 0x7fc1740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.215+0000 7fc17effd640 1 -- 172.21.15.17:0/1865167529 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fc1840cf230 con 0x7fc1881017b0 2024-03-24T03:00:49.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.216+0000 7fc18de1d640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc1740a2800 0x7fc1740a4cc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc188101610 tx=0x7fc178005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.220+0000 7fc17effd640 1 -- 172.21.15.17:0/1865167529 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc18409b840 con 0x7fc1881017b0 2024-03-24T03:00:49.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:48 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1272372154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:00:49.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:48 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4092164923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:00:49.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:48 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/518916820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:00:49.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:48 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1272372154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:00:49.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:48 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4092164923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:00:49.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:48 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/518916820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:00:49.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.405+0000 7fc190081640 1 -- 172.21.15.17:0/1865167529 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1881a07b0 con 0x7fc1881017b0 2024-03-24T03:00:49.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.407+0000 7fc17effd640 1 -- 172.21.15.17:0/1865167529 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24955 (secure 0 0 0) 0x7fc18409f8e0 con 0x7fc1881017b0 2024-03-24T03:00:49.407 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:00:49.407 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":93,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T03:00:31.817661+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T03:00:49.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.409+0000 7fc190081640 1 -- 172.21.15.17:0/1865167529 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc1740a2800 msgr2=0x7fc1740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.409+0000 7fc190081640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc1740a2800 0x7fc1740a4cc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc188101610 tx=0x7fc178005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:49.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.409+0000 7fc190081640 1 -- 172.21.15.17:0/1865167529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1881017b0 msgr2=0x7fc188197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.409+0000 7fc190081640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1881017b0 0x7fc188197ef0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e9f0 tx=0x7fc18400eeb0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.409+0000 7fc190081640 1 -- 172.21.15.17:0/1865167529 shutdown_connections 2024-03-24T03:00:49.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.409+0000 7fc190081640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fc1740a2800 0x7fc1740a4cc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.409+0000 7fc190081640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc188102170 0x7fc18819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.409+0000 7fc190081640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1881017b0 0x7fc188197ef0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.409+0000 7fc190081640 1 --2- 172.21.15.17:0/1865167529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1881005c0 0x7fc1881979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.409+0000 7fc190081640 1 -- 172.21.15.17:0/1865167529 >> 172.21.15.17:0/1865167529 conn(0x7fc1880fbb30 msgr2=0x7fc1880fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.409+0000 7fc190081640 1 -- 172.21.15.17:0/1865167529 shutdown_connections 2024-03-24T03:00:49.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.409+0000 7fc190081640 1 -- 172.21.15.17:0/1865167529 wait complete. 2024-03-24T03:00:49.421 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-24T02:58:36.476051+0000', 'modified': '2024-03-24T02:58:59.074424+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24529, 'mds_1': 24521, 'mds_2': 24511, 'mds_3': 24493, 'mds_4': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.17:6839/2895019504', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6838', 'nonce': 2895019504}, {'type': 'v1', 'addr': '172.21.15.17:6839', 'nonce': 2895019504}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.17:6841/1592961709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6840', 'nonce': 1592961709}, {'type': 'v1', 'addr': '172.21.15.17:6841', 'nonce': 1592961709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24419': {'gid': 24419, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6833/3469034636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 3469034636}, {'type': 'v1', 'addr': '172.21.15.19:6833', 'nonce': 3469034636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6833/920982807', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 920982807}, {'type': 'v1', 'addr': '172.21.15.188:6833', 'nonce': 920982807}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/1025006466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 1025006466}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 1025006466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.19:6837/2540260543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6836', 'nonce': 2540260543}, {'type': 'v1', 'addr': '172.21.15.19:6837', 'nonce': 2540260543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.188:6835/3649219818', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 3649219818}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 3649219818}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.188:6837/3512350624', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 3512350624}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 3512350624}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.19:6839/2102503456', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 2102503456}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 2102503456}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.188:6839/81409664', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 81409664}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 81409664}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24529, 'qdb_cluster': [24511, 24529, 24485, 24493, 24521]} 2024-03-24T03:00:49.421 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-24T03:00:49.421 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.525+0000 7f25e7737640 1 -- 172.21.15.17:0/3734192525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e0102140 msgr2=0x7f25e0104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.525+0000 7f25e7737640 1 --2- 172.21.15.17:0/3734192525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e0102140 0x7f25e0104530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009860 tx=0x7f25dc02bb70 comp rx=0 tx=0).stop 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.525+0000 7f25e7737640 1 -- 172.21.15.17:0/3734192525 shutdown_connections 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.525+0000 7f25e7737640 1 --2- 172.21.15.17:0/3734192525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25e01073a0 0x7f25e01097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.525+0000 7f25e7737640 1 --2- 172.21.15.17:0/3734192525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e0104a70 0x7f25e0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.525+0000 7f25e7737640 1 --2- 172.21.15.17:0/3734192525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e0102140 0x7f25e0104530 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.525+0000 7f25e7737640 1 -- 172.21.15.17:0/3734192525 >> 172.21.15.17:0/3734192525 conn(0x7f25e00fbb50 msgr2=0x7f25e00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.525+0000 7f25e7737640 1 -- 172.21.15.17:0/3734192525 shutdown_connections 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.525+0000 7f25e7737640 1 -- 172.21.15.17:0/3734192525 wait complete. 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e7737640 1 Processor -- start 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e7737640 1 -- start start 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e7737640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e0102140 0x7f25e0101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e7737640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e0104a70 0x7f25e01000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e7737640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25e01073a0 0x7f25e0100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e7737640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e010d350 con 0x7f25e0104a70 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e7737640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e010d1d0 con 0x7f25e0102140 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e7737640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e010d4d0 con 0x7f25e01073a0 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e4cd2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e0104a70 0x7f25e01000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e54d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e0102140 0x7f25e0101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e5cd4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25e01073a0 0x7f25e0100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e4cd2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e0104a70 0x7f25e01000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41892/0 (socket says 172.21.15.17:41892) 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e54d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e0102140 0x7f25e0101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:41224/0 (socket says 172.21.15.17:41224) 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.526+0000 7f25e4cd2640 1 -- 172.21.15.17:0/2031663615 learned_addr learned my addr 172.21.15.17:0/2031663615 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.527+0000 7f25e5cd4640 1 -- 172.21.15.17:0/2031663615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e0102140 msgr2=0x7f25e0101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.527+0000 7f25e5cd4640 1 --2- 172.21.15.17:0/2031663615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e0102140 0x7f25e0101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.527+0000 7f25e5cd4640 1 -- 172.21.15.17:0/2031663615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e0104a70 msgr2=0x7f25e01000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.527+0000 7f25e5cd4640 1 --2- 172.21.15.17:0/2031663615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e0104a70 0x7f25e01000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.527+0000 7f25e5cd4640 1 -- 172.21.15.17:0/2031663615 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e0100ec0 con 0x7f25e01073a0 2024-03-24T03:00:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.527+0000 7f25e5cd4640 1 --2- 172.21.15.17:0/2031663615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25e01073a0 0x7f25e0100630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009fd0 tx=0x7f25d800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.527+0000 7f25d67fc640 1 -- 172.21.15.17:0/2031663615 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d800cd60 con 0x7f25e01073a0 2024-03-24T03:00:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.527+0000 7f25e7737640 1 -- 172.21.15.17:0/2031663615 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e01a1b70 con 0x7f25e01073a0 2024-03-24T03:00:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.527+0000 7f25d67fc640 1 -- 172.21.15.17:0/2031663615 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25d8005020 con 0x7f25e01073a0 2024-03-24T03:00:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.527+0000 7f25d67fc640 1 -- 172.21.15.17:0/2031663615 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8010690 con 0x7f25e01073a0 2024-03-24T03:00:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.527+0000 7f25e7737640 1 -- 172.21.15.17:0/2031663615 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e01a20c0 con 0x7f25e01073a0 2024-03-24T03:00:49.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.529+0000 7f25d67fc640 1 -- 172.21.15.17:0/2031663615 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f25d8005540 con 0x7f25e01073a0 2024-03-24T03:00:49.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.530+0000 7f25d67fc640 1 --2- 172.21.15.17:0/2031663615 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25cc0a2800 0x7f25cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.530+0000 7f25d67fc640 1 -- 172.21.15.17:0/2031663615 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f25d80cf5e0 con 0x7f25e01073a0 2024-03-24T03:00:49.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.530+0000 7f25e54d3640 1 --2- 172.21.15.17:0/2031663615 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25cc0a2800 0x7f25cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25e54d3640 1 --2- 172.21.15.17:0/2031663615 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25cc0a2800 0x7f25cc0a4cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f25dc02c180 tx=0x7f25dc034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25ea3fc740 1 Processor -- start 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25ea3fc740 1 -- start start 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25ea3fc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156e740 0x55950156eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25ea3fc740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156f130 0x559501573c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25ea3fc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 0x559501576730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25ea3fc740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595014d3c20 con 0x55950156f130 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25ea3fc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950147ca40 con 0x55950156e740 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25ea3fc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595014cfcc0 con 0x5595015742a0 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25e4cd2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156f130 0x559501573c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25e54d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156e740 0x55950156eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25e5cd4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 0x559501576730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25e54d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156e740 0x55950156eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:41230/0 (socket says 172.21.15.17:41230) 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.531+0000 7f25e54d3640 1 -- 172.21.15.17:0/2009591760 learned_addr learned my addr 172.21.15.17:0/2009591760 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.532+0000 7f25e5cd4640 1 -- 172.21.15.17:0/2009591760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156e740 msgr2=0x55950156eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.532+0000 7f25e5cd4640 1 --2- 172.21.15.17:0/2009591760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156e740 0x55950156eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.532+0000 7f25e5cd4640 1 -- 172.21.15.17:0/2009591760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156f130 msgr2=0x559501573c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.532+0000 7f25e5cd4640 1 --2- 172.21.15.17:0/2009591760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156f130 0x559501573c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.532+0000 7f25e5cd4640 1 -- 172.21.15.17:0/2009591760 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55950149ef90 con 0x5595015742a0 2024-03-24T03:00:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.532+0000 7f25e4cd2640 1 --2- 172.21.15.17:0/2009591760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156f130 0x559501573c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.532+0000 7f25e5cd4640 1 --2- 172.21.15.17:0/2009591760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 0x559501576730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f25d809b7e0 tx=0x7f25d809b750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e67520a30d9296a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.532+0000 7f25cb7fe640 1 -- 172.21.15.17:0/2009591760 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80cf070 con 0x5595015742a0 2024-03-24T03:00:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.532+0000 7f25cb7fe640 1 -- 172.21.15.17:0/2009591760 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25d8099b80 con 0x5595015742a0 2024-03-24T03:00:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.532+0000 7f25cb7fe640 1 -- 172.21.15.17:0/2009591760 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80a3df0 con 0x5595015742a0 2024-03-24T03:00:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.533+0000 7f25ea3fc740 1 -- 172.21.15.17:0/2009591760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 msgr2=0x559501576730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.533+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/2009591760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 0x559501576730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f25d809b7e0 tx=0x7f25d809b750 comp rx=0 tx=0).stop 2024-03-24T03:00:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.533+0000 7f25ea3fc740 1 -- 172.21.15.17:0/2009591760 shutdown_connections 2024-03-24T03:00:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.533+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/2009591760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 0x559501576730 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.533+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/2009591760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156f130 0x559501573c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.533+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/2009591760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156e740 0x55950156eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.533+0000 7f25ea3fc740 1 -- 172.21.15.17:0/2009591760 >> 172.21.15.17:0/2009591760 conn(0x559501536a60 msgr2=0x55950156a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.533+0000 7f25ea3fc740 1 -- 172.21.15.17:0/2009591760 shutdown_connections 2024-03-24T03:00:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.533+0000 7f25ea3fc740 1 -- 172.21.15.17:0/2009591760 wait complete. 2024-03-24T03:00:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.533+0000 7f25ea3fc740 1 -- start start 2024-03-24T03:00:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.533+0000 7f25ea3fc740 1 Processor -- start 2024-03-24T03:00:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25ea3fc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156e740 0x559501578cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25ea3fc740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156f130 0x5595015791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25ea3fc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 0x559501580230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25ea3fc740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559501577250 con 0x55950156f130 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25ea3fc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950148fe90 con 0x5595015742a0 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25ea3fc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595015773d0 con 0x55950156e740 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25ea3fc740 10 client.0 _ll_register_callbacks cb 0x5595011e7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e5cd4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 0x559501580230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e54d3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156e740 0x559501578cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e5cd4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 0x559501580230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:41240/0 (socket says 172.21.15.17:41240) 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e4cd2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156f130 0x5595015791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e5cd4640 1 -- 172.21.15.17:0/4238674482 learned_addr learned my addr 172.21.15.17:0/4238674482 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e54d3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156e740 0x559501578cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:42940/0 (socket says 172.21.15.17:42940) 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e54d3640 1 -- 172.21.15.17:0/4238674482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 msgr2=0x559501580230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e54d3640 1 --2- 172.21.15.17:0/4238674482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 0x559501580230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e54d3640 1 -- 172.21.15.17:0/4238674482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156f130 msgr2=0x5595015791f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e54d3640 1 --2- 172.21.15.17:0/4238674482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156f130 0x5595015791f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e54d3640 1 -- 172.21.15.17:0/4238674482 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55950149c9d0 con 0x55950156e740 2024-03-24T03:00:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e5cd4640 1 --2- 172.21.15.17:0/4238674482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 0x559501580230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25e54d3640 1 --2- 172.21.15.17:0/4238674482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156e740 0x559501578cb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0057c0 tx=0x7f25dc02bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3abe000834f4710 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.534+0000 7f25c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25c8ff9640 1 -- 172.21.15.17:0/4238674482 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc04f070 con 0x55950156e740 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25c8ff9640 1 -- 172.21.15.17:0/4238674482 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f25dc02c920 con 0x55950156e740 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25ea3fc740 1 -- 172.21.15.17:0/4238674482 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25d8099b80 con 0x55950156e740 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25c8ff9640 1 -- 172.21.15.17:0/4238674482 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25dc05b050 con 0x55950156e740 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25ea3fc740 10 client.25330 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25ea3fc740 1 -- 172.21.15.17:0/4238674482 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5595011e82d0 con 0x55950156e740 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25c8ff9640 1 -- 172.21.15.17:0/4238674482 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25d8099b80 con 0x55950156e740 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25c8ff9640 1 -- 172.21.15.17:0/4238674482 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f25dc003680 con 0x55950156e740 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25ea3fc740 10 client.25330 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25ea3fc740 10 client.25330 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25ea3fc740 20 client.25330 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/4238674482 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x559501585410 0x559501584520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25ea3fc740 4 client.25330 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-24T03:00:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.535+0000 7f25ea3fc740 1 -- 172.21.15.17:0/4238674482 --> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595014e1160 con 0x559501585410 2024-03-24T03:00:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.536+0000 7f25e4cd2640 1 --2- 172.21.15.17:0/4238674482 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x559501585410 0x559501584520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.536+0000 7f25e4cd2640 1 --2- 172.21.15.17:0/4238674482 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x559501585410 0x559501584520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9bddaea27e740de5 server_cookie=e76ea8d506658c19 in_seq=0 out_seq=0 2024-03-24T03:00:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.536+0000 7f25c8ff9640 10 client.25330 ms_handle_connect on v2:172.21.15.19:6836/2540260543 2024-03-24T03:00:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25c8ff9640 1 -- 172.21.15.17:0/4238674482 <== mds.4 v2:172.21.15.19:6836/2540260543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5595014e1160 con 0x559501585410 2024-03-24T03:00:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25c8ff9640 10 client.25330 handle_command_reply: tid=0 2024-03-24T03:00:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25ea3fc740 1 client.25330 shutdown 2024-03-24T03:00:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25ea3fc740 1 -- 172.21.15.17:0/4238674482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156e740 msgr2=0x559501578cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/4238674482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156e740 0x559501578cb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0057c0 tx=0x7f25dc02bb70 comp rx=0 tx=0).stop 2024-03-24T03:00:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25ea3fc740 1 -- 172.21.15.17:0/4238674482 shutdown_connections 2024-03-24T03:00:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/4238674482 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x559501585410 0x559501584520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/4238674482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 0x559501580230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/4238674482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156f130 0x5595015791f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/4238674482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156e740 0x559501578cb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25c8ff9640 0 client.25330 ms_handle_reset on v2:172.21.15.19:6836/2540260543 2024-03-24T03:00:49.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25ea3fc740 1 -- 172.21.15.17:0/4238674482 >> 172.21.15.17:0/4238674482 conn(0x559501536a60 msgr2=0x5595015692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25ea3fc740 1 -- 172.21.15.17:0/4238674482 shutdown_connections 2024-03-24T03:00:49.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25ea3fc740 1 -- 172.21.15.17:0/4238674482 wait complete. 2024-03-24T03:00:49.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.537+0000 7f25ea3fc740 20 client.25330 trim_cache size 0 max 16384 2024-03-24T03:00:49.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.555+0000 7f25ea3fc740 1 Processor -- start 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.555+0000 7f25ea3fc740 1 -- start start 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.555+0000 7f25ea3fc740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156e740 0x559501569bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.555+0000 7f25ea3fc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156f130 0x5595015801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.555+0000 7f25ea3fc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 0x55950156a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.555+0000 7f25ea3fc740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559501584c30 con 0x55950156e740 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.555+0000 7f25ea3fc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559501585290 con 0x55950156f130 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.555+0000 7f25ea3fc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595015858f0 con 0x5595015742a0 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e4cd2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156f130 0x5595015801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e5cd4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 0x55950156a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e4cd2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156f130 0x5595015801c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:41244/0 (socket says 172.21.15.17:41244) 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e5cd4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 0x55950156a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:42942/0 (socket says 172.21.15.17:42942) 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e4cd2640 1 -- 172.21.15.17:0/1001362357 learned_addr learned my addr 172.21.15.17:0/1001362357 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e54d3640 1 --2- 172.21.15.17:0/1001362357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156e740 0x559501569bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e4cd2640 1 -- 172.21.15.17:0/1001362357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 msgr2=0x55950156a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e4cd2640 1 --2- 172.21.15.17:0/1001362357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 0x55950156a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e4cd2640 1 -- 172.21.15.17:0/1001362357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156e740 msgr2=0x559501569bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e4cd2640 1 --2- 172.21.15.17:0/1001362357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156e740 0x559501569bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e4cd2640 1 -- 172.21.15.17:0/1001362357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595012b26b0 con 0x55950156f130 2024-03-24T03:00:49.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e5cd4640 1 --2- 172.21.15.17:0/1001362357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 0x55950156a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:49.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25e4cd2640 1 --2- 172.21.15.17:0/1001362357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156f130 0x5595015801c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f25d0000c00 tx=0x7f25d0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5964f6cf017ca721 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25caffd640 1 -- 172.21.15.17:0/1001362357 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0025070 con 0x55950156f130 2024-03-24T03:00:49.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25caffd640 1 -- 172.21.15.17:0/1001362357 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25d0020c50 con 0x55950156f130 2024-03-24T03:00:49.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.556+0000 7f25caffd640 1 -- 172.21.15.17:0/1001362357 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0010970 con 0x55950156f130 2024-03-24T03:00:49.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.557+0000 7f25ea3fc740 1 -- 172.21.15.17:0/1001362357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156f130 msgr2=0x5595015801c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.557+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/1001362357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156f130 0x5595015801c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f25d0000c00 tx=0x7f25d0009260 comp rx=0 tx=0).stop 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.557+0000 7f25ea3fc740 1 -- 172.21.15.17:0/1001362357 shutdown_connections 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.557+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/1001362357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595015742a0 0x55950156a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.557+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/1001362357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55950156f130 0x5595015801c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.557+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/1001362357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156e740 0x559501569bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.557+0000 7f25ea3fc740 1 -- 172.21.15.17:0/1001362357 >> 172.21.15.17:0/1001362357 conn(0x559501536a60 msgr2=0x55950156fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.557+0000 7f25ea3fc740 1 -- 172.21.15.17:0/1001362357 shutdown_connections 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.557+0000 7f25ea3fc740 1 -- 172.21.15.17:0/1001362357 wait complete. 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25ea3fc740 1 -- start start 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25ea3fc740 1 Processor -- start 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25ea3fc740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156e740 0x55950157fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25ea3fc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156f130 0x559501580210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25ea3fc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 0x55950157c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25ea3fc740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595015865b0 con 0x55950156e740 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25ea3fc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559501585f50 con 0x5595015742a0 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25ea3fc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559501586c10 con 0x55950156f130 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25ea3fc740 10 client.0 _ll_register_callbacks cb 0x7f25d80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25e5cd4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 0x55950157c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25e54d3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156e740 0x55950157fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25e4cd2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156f130 0x559501580210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25e4cd2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156f130 0x559501580210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:42956/0 (socket says 172.21.15.17:42956) 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25e5cd4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 0x55950157c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:41252/0 (socket says 172.21.15.17:41252) 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25e54d3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156e740 0x55950157fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41932/0 (socket says 172.21.15.17:41932) 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.558+0000 7f25e4cd2640 1 -- 172.21.15.17:0/178894721 learned_addr learned my addr 172.21.15.17:0/178894721 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25e4cd2640 1 -- 172.21.15.17:0/178894721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 msgr2=0x55950157c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25e4cd2640 1 --2- 172.21.15.17:0/178894721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 0x55950157c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25e4cd2640 1 -- 172.21.15.17:0/178894721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156e740 msgr2=0x55950157fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25e4cd2640 1 --2- 172.21.15.17:0/178894721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156e740 0x55950157fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25e4cd2640 1 -- 172.21.15.17:0/178894721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55950157d090 con 0x55950156f130 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25e54d3640 1 --2- 172.21.15.17:0/178894721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156e740 0x55950157fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25e4cd2640 1 --2- 172.21.15.17:0/178894721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156f130 0x559501580210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f25d0020e80 tx=0x7f25d00104d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa7f07c769b72940 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25c8ff9640 1 -- 172.21.15.17:0/178894721 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0025040 con 0x55950156f130 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25ea3fc740 1 -- 172.21.15.17:0/178894721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25d0020c50 con 0x55950156f130 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25c8ff9640 1 -- 172.21.15.17:0/178894721 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f25d00200f0 con 0x55950156f130 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25c8ff9640 1 -- 172.21.15.17:0/178894721 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25d0038050 con 0x55950156f130 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25ea3fc740 10 client.25336 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.559+0000 7f25ea3fc740 1 -- 172.21.15.17:0/178894721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5595015e1300 con 0x55950156f130 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.560+0000 7f25c8ff9640 1 -- 172.21.15.17:0/178894721 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25d0020c50 con 0x55950156f130 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.560+0000 7f25c8ff9640 1 -- 172.21.15.17:0/178894721 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f25d0004830 con 0x55950156f130 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.560+0000 7f25ea3fc740 10 client.25336 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.560+0000 7f25ea3fc740 10 client.25336 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.560+0000 7f25ea3fc740 20 client.25336 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.560+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/178894721 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x5595015e5650 0x5595015e15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.560+0000 7f25ea3fc740 4 client.25336 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.560+0000 7f25ea3fc740 1 -- 172.21.15.17:0/178894721 --> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5595014db4b0 con 0x5595015e5650 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.560+0000 7f25e54d3640 1 --2- 172.21.15.17:0/178894721 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x5595015e5650 0x5595015e15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.560+0000 7f25c8ff9640 10 client.25336 ms_handle_connect on v2:172.21.15.19:6836/2540260543 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.560+0000 7f25e54d3640 1 --2- 172.21.15.17:0/178894721 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x5595015e5650 0x5595015e15f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d8c0a41d3bb6f19 server_cookie=995086918625c35d in_seq=0 out_seq=0 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25c8ff9640 1 -- 172.21.15.17:0/178894721 <== mds.4 v2:172.21.15.19:6836/2540260543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5595014db4b0 con 0x5595015e5650 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25c8ff9640 10 client.25336 handle_command_reply: tid=0 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25ea3fc740 1 client.25336 shutdown 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25ea3fc740 1 -- 172.21.15.17:0/178894721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156f130 msgr2=0x559501580210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/178894721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156f130 0x559501580210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f25d0020e80 tx=0x7f25d00104d0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25ea3fc740 1 -- 172.21.15.17:0/178894721 shutdown_connections 2024-03-24T03:00:49.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/178894721 >> [v2:172.21.15.19:6836/2540260543,v1:172.21.15.19:6837/2540260543] conn(0x5595015e5650 0x5595015e15f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/178894721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595015742a0 0x55950157c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25c8ff9640 0 client.25336 ms_handle_reset on v2:172.21.15.19:6836/2540260543 2024-03-24T03:00:49.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/178894721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55950156f130 0x559501580210 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25ea3fc740 1 --2- 172.21.15.17:0/178894721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55950156e740 0x55950157fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25ea3fc740 1 -- 172.21.15.17:0/178894721 >> 172.21.15.17:0/178894721 conn(0x559501536a60 msgr2=0x559501569bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25ea3fc740 1 -- 172.21.15.17:0/178894721 shutdown_connections 2024-03-24T03:00:49.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25ea3fc740 1 -- 172.21.15.17:0/178894721 wait complete. 2024-03-24T03:00:49.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.561+0000 7f25ea3fc740 20 client.25336 trim_cache size 0 max 16384 2024-03-24T03:00:49.567 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:00:49.567 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T03:00:49.567 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 4, 2024-03-24T03:00:49.567 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24485, 2024-03-24T03:00:49.568 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-24T03:00:49.568 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-24T03:00:49.568 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-24T03:00:49.568 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 111.490359557, 2024-03-24T03:00:49.568 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 28, 2024-03-24T03:00:49.568 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 93, 2024-03-24T03:00:49.568 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-03-24T03:00:49.568 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 150.00126887499999 2024-03-24T03:00:49.568 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:00:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.562+0000 7f25e7737640 1 -- 172.21.15.17:0/2031663615 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25cc0a2800 msgr2=0x7f25cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.562+0000 7f25e7737640 1 --2- 172.21.15.17:0/2031663615 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25cc0a2800 0x7f25cc0a4cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f25dc02c180 tx=0x7f25dc034ad0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.562+0000 7f25e7737640 1 -- 172.21.15.17:0/2031663615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25e01073a0 msgr2=0x7f25e0100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.562+0000 7f25e7737640 1 --2- 172.21.15.17:0/2031663615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25e01073a0 0x7f25e0100630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009fd0 tx=0x7f25d800ee50 comp rx=0 tx=0).stop 2024-03-24T03:00:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.562+0000 7f25e7737640 1 -- 172.21.15.17:0/2031663615 shutdown_connections 2024-03-24T03:00:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.562+0000 7f25e7737640 1 --2- 172.21.15.17:0/2031663615 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f25cc0a2800 0x7f25cc0a4cc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.562+0000 7f25e7737640 1 --2- 172.21.15.17:0/2031663615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25e01073a0 0x7f25e0100630 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.562+0000 7f25e7737640 1 --2- 172.21.15.17:0/2031663615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e0104a70 0x7f25e01000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.562+0000 7f25e7737640 1 --2- 172.21.15.17:0/2031663615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e0102140 0x7f25e0101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.562+0000 7f25e7737640 1 -- 172.21.15.17:0/2031663615 >> 172.21.15.17:0/2031663615 conn(0x7f25e00fbb50 msgr2=0x7f25e0102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.562+0000 7f25e7737640 1 -- 172.21.15.17:0/2031663615 shutdown_connections 2024-03-24T03:00:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.562+0000 7f25e7737640 1 -- 172.21.15.17:0/2031663615 wait complete. 2024-03-24T03:00:49.574 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-24T03:00:49.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.678+0000 7fd2c413b640 1 -- 172.21.15.17:0/696152375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2bc1016c0 msgr2=0x7fd2bc101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.678+0000 7fd2c413b640 1 --2- 172.21.15.17:0/696152375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2bc1016c0 0x7fd2bc101ae0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009900 tx=0x7fd2b402bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:49.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.678+0000 7fd2c413b640 1 -- 172.21.15.17:0/696152375 shutdown_connections 2024-03-24T03:00:49.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.678+0000 7fd2c413b640 1 --2- 172.21.15.17:0/696152375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2bc103270 0x7fd2bc109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.678+0000 7fd2c413b640 1 --2- 172.21.15.17:0/696152375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2bc1028b0 0x7fd2bc102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.678+0000 7fd2c413b640 1 --2- 172.21.15.17:0/696152375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2bc1016c0 0x7fd2bc101ae0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.678+0000 7fd2c413b640 1 -- 172.21.15.17:0/696152375 >> 172.21.15.17:0/696152375 conn(0x7fd2bc0fcc30 msgr2=0x7fd2bc0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.678+0000 7fd2c413b640 1 -- 172.21.15.17:0/696152375 shutdown_connections 2024-03-24T03:00:49.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.678+0000 7fd2c413b640 1 -- 172.21.15.17:0/696152375 wait complete. 2024-03-24T03:00:49.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.678+0000 7fd2c413b640 1 Processor -- start 2024-03-24T03:00:49.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.678+0000 7fd2c413b640 1 -- start start 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c413b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2bc1016c0 0x7fd2bc1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c413b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2bc1028b0 0x7fd2bc197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c413b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2bc103270 0x7fd2bc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c413b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc10d390 con 0x7fd2bc103270 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c413b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc10d210 con 0x7fd2bc1028b0 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c413b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc10d510 con 0x7fd2bc1016c0 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c26d8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2bc103270 0x7fd2bc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c1ed7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2bc1016c0 0x7fd2bc1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c26d8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2bc103270 0x7fd2bc19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41946/0 (socket says 172.21.15.17:41946) 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c26d8640 1 -- 172.21.15.17:0/113085155 learned_addr learned my addr 172.21.15.17:0/113085155 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2bc1028b0 0x7fd2bc197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c26d8640 1 -- 172.21.15.17:0/113085155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2bc1016c0 msgr2=0x7fd2bc1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c26d8640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2bc1016c0 0x7fd2bc1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c26d8640 1 -- 172.21.15.17:0/113085155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2bc1028b0 msgr2=0x7fd2bc197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c26d8640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2bc1028b0 0x7fd2bc197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c26d8640 1 -- 172.21.15.17:0/113085155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc19f5f0 con 0x7fd2bc103270 2024-03-24T03:00:49.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c1ed7640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2bc1016c0 0x7fd2bc1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:49.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c26d8640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2bc103270 0x7fd2bc19ef80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00eb10 tx=0x7fd2ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2baffd640 1 -- 172.21.15.17:0/113085155 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac004b20 con 0x7fd2bc103270 2024-03-24T03:00:49.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2baffd640 1 -- 172.21.15.17:0/113085155 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2ac004cc0 con 0x7fd2bc103270 2024-03-24T03:00:49.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c413b640 1 -- 172.21.15.17:0/113085155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2bc19f8e0 con 0x7fd2bc103270 2024-03-24T03:00:49.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2baffd640 1 -- 172.21.15.17:0/113085155 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac010760 con 0x7fd2bc103270 2024-03-24T03:00:49.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.679+0000 7fd2c413b640 1 -- 172.21.15.17:0/113085155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2bc19fe30 con 0x7fd2bc103270 2024-03-24T03:00:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.682+0000 7fd2baffd640 1 -- 172.21.15.17:0/113085155 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd2ac005020 con 0x7fd2bc103270 2024-03-24T03:00:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.682+0000 7fd2c6e00740 1 Processor -- start 2024-03-24T03:00:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.682+0000 7fd2c6e00740 1 -- start start 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c6e00740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6c9b50 0x56467b6c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c6e00740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6ca540 0x56467b6cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c6e00740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 0x56467b6d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c6e00740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467b62ac20 con 0x56467b6cf6b0 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c6e00740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467b5d3a40 con 0x56467b6ca540 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c6e00740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467b626cc0 con 0x56467b6c9b50 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c16d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 0x56467b6d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c1ed7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6ca540 0x56467b6cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2baffd640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd2a40a2800 0x7fd2a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c16d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 0x56467b6d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41948/0 (socket says 172.21.15.17:41948) 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c16d6640 1 -- 172.21.15.17:0/1435114606 learned_addr learned my addr 172.21.15.17:0/1435114606 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/1435114606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6c9b50 0x56467b6c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c26d8640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd2a40a2800 0x7fd2a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2baffd640 1 -- 172.21.15.17:0/113085155 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fd2ac0cf580 con 0x7fd2bc103270 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c16d6640 1 -- 172.21.15.17:0/1435114606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6c9b50 msgr2=0x56467b6c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/1435114606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6c9b50 0x56467b6c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c16d6640 1 -- 172.21.15.17:0/1435114606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6ca540 msgr2=0x56467b6cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/1435114606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6ca540 0x56467b6cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c16d6640 1 -- 172.21.15.17:0/1435114606 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56467b5f5f90 con 0x56467b6cf6b0 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c1ed7640 1 --2- 172.21.15.17:0/1435114606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6ca540 0x56467b6cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.683+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/1435114606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 0x56467b6d1b40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009fd0 tx=0x7fd2b0009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb2ee79afc42e11 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2c26d8640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd2a40a2800 0x7fd2a40a4cc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0984b0 tx=0x7fd2ac01bb30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2a3fff640 1 -- 172.21.15.17:0/1435114606 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000f790 con 0x56467b6cf6b0 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2a3fff640 1 -- 172.21.15.17:0/1435114606 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2b000fe10 con 0x56467b6cf6b0 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2a3fff640 1 -- 172.21.15.17:0/1435114606 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b001fd60 con 0x56467b6cf6b0 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2c6e00740 1 -- 172.21.15.17:0/1435114606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 msgr2=0x56467b6d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/1435114606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 0x56467b6d1b40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009fd0 tx=0x7fd2b0009370 comp rx=0 tx=0).stop 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2c6e00740 1 -- 172.21.15.17:0/1435114606 shutdown_connections 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/1435114606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 0x56467b6d1b40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/1435114606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6ca540 0x56467b6cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/1435114606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6c9b50 0x56467b6c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2c6e00740 1 -- 172.21.15.17:0/1435114606 >> 172.21.15.17:0/1435114606 conn(0x56467b68da60 msgr2=0x56467b6c54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2c6e00740 1 -- 172.21.15.17:0/1435114606 shutdown_connections 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.684+0000 7fd2c6e00740 1 -- 172.21.15.17:0/1435114606 wait complete. 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c6e00740 1 -- start start 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c6e00740 1 Processor -- start 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c6e00740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6c9b50 0x56467b6d4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c6e00740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6ca540 0x56467b6d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c6e00740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6cf6b0 0x56467b6db5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c6e00740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467b6d2660 con 0x56467b6ca540 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c6e00740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467b5e6e90 con 0x56467b6cf6b0 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c6e00740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467b6d27e0 con 0x56467b6c9b50 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c6e00740 10 client.0 _ll_register_callbacks cb 0x56467b33eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c1ed7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6ca540 0x56467b6d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c16d6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6c9b50 0x56467b6d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c1ed7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6ca540 0x56467b6d45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41950/0 (socket says 172.21.15.17:41950) 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c16d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6cf6b0 0x56467b6db5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.685+0000 7fd2c1ed7640 1 -- 172.21.15.17:0/3645084935 learned_addr learned my addr 172.21.15.17:0/3645084935 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2c16d6640 1 -- 172.21.15.17:0/3645084935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6c9b50 msgr2=0x56467b6d4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/3645084935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6c9b50 0x56467b6d4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2c16d6640 1 -- 172.21.15.17:0/3645084935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6ca540 msgr2=0x56467b6d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/3645084935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6ca540 0x56467b6d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2c16d6640 1 -- 172.21.15.17:0/3645084935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56467b5f39d0 con 0x56467b6cf6b0 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2c1ed7640 1 --2- 172.21.15.17:0/3645084935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6ca540 0x56467b6d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/3645084935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6cf6b0 0x56467b6db5e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0021fd0 tx=0x7fd2b0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9c2134a81fc855c server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2a17fa640 1 -- 172.21.15.17:0/3645084935 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b001fb60 con 0x56467b6cf6b0 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2c6e00740 1 -- 172.21.15.17:0/3645084935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2b000fe10 con 0x56467b6cf6b0 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2a17fa640 1 -- 172.21.15.17:0/3645084935 <== mon.2 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fd2b0019070 con 0x56467b6cf6b0 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2a17fa640 1 -- 172.21.15.17:0/3645084935 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2b0035050 con 0x56467b6cf6b0 2024-03-24T03:00:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2c6e00740 10 client.25235 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2c6e00740 1 -- 172.21.15.17:0/3645084935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56467b33f2d0 con 0x56467b6cf6b0 2024-03-24T03:00:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.686+0000 7fd2a17fa640 1 -- 172.21.15.17:0/3645084935 <== mon.2 v2:172.21.15.19:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2b000fe10 con 0x56467b6cf6b0 2024-03-24T03:00:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.687+0000 7fd2a17fa640 1 -- 172.21.15.17:0/3645084935 <== mon.2 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd2b001d020 con 0x56467b6cf6b0 2024-03-24T03:00:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.687+0000 7fd2c6e00740 10 client.25235 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.687+0000 7fd2c6e00740 10 client.25235 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-24T03:00:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.687+0000 7fd2c6e00740 20 client.25235 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.687+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/3645084935 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x56467b6e04d0 0x56467b6df740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.687+0000 7fd2c6e00740 4 client.25235 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-24T03:00:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.687+0000 7fd2c6e00740 1 -- 172.21.15.17:0/3645084935 --> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56467b638160 con 0x56467b6e04d0 2024-03-24T03:00:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.687+0000 7fd2c1ed7640 1 --2- 172.21.15.17:0/3645084935 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x56467b6e04d0 0x56467b6df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.687+0000 7fd2c1ed7640 1 --2- 172.21.15.17:0/3645084935 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x56467b6e04d0 0x56467b6df740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3f3a18c58178fdbe server_cookie=b45dffc2bd0f7d92 in_seq=0 out_seq=0 2024-03-24T03:00:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.687+0000 7fd2a17fa640 10 client.25235 ms_handle_connect on v2:172.21.15.188:6834/3649219818 2024-03-24T03:00:49.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2a17fa640 1 -- 172.21.15.17:0/3645084935 <== mds.3 v2:172.21.15.188:6834/3649219818 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56467b638160 con 0x56467b6e04d0 2024-03-24T03:00:49.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2a17fa640 10 client.25235 handle_command_reply: tid=0 2024-03-24T03:00:49.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2c6e00740 1 client.25235 shutdown 2024-03-24T03:00:49.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2c6e00740 1 -- 172.21.15.17:0/3645084935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6cf6b0 msgr2=0x56467b6db5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/3645084935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6cf6b0 0x56467b6db5e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0021fd0 tx=0x7fd2b0005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:49.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2c6e00740 1 -- 172.21.15.17:0/3645084935 shutdown_connections 2024-03-24T03:00:49.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/3645084935 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x56467b6e04d0 0x56467b6df740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/3645084935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6cf6b0 0x56467b6db5e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/3645084935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6ca540 0x56467b6d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2a17fa640 0 client.25235 ms_handle_reset on v2:172.21.15.188:6834/3649219818 2024-03-24T03:00:49.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/3645084935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6c9b50 0x56467b6d4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2c6e00740 1 -- 172.21.15.17:0/3645084935 >> 172.21.15.17:0/3645084935 conn(0x56467b68da60 msgr2=0x56467b6c02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.688+0000 7fd2c6e00740 1 -- 172.21.15.17:0/3645084935 shutdown_connections 2024-03-24T03:00:49.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.689+0000 7fd2c6e00740 1 -- 172.21.15.17:0/3645084935 wait complete. 2024-03-24T03:00:49.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.689+0000 7fd2c6e00740 20 client.25235 trim_cache size 0 max 16384 2024-03-24T03:00:49.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c6e00740 1 Processor -- start 2024-03-24T03:00:49.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c6e00740 1 -- start start 2024-03-24T03:00:49.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c6e00740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6c9b50 0x56467b6d3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c6e00740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6ca540 0x56467b6db500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c6e00740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 0x56467b6d41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c6e00740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467b6dfcf0 con 0x56467b6cf6b0 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c6e00740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467b6e0350 con 0x56467b6c9b50 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c6e00740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467b6e09b0 con 0x56467b6ca540 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c16d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 0x56467b6d41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c16d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 0x56467b6d41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41954/0 (socket says 172.21.15.17:41954) 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c16d6640 1 -- 172.21.15.17:0/489038089 learned_addr learned my addr 172.21.15.17:0/489038089 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/489038089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6c9b50 0x56467b6d3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c1ed7640 1 --2- 172.21.15.17:0/489038089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6ca540 0x56467b6db500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c16d6640 1 -- 172.21.15.17:0/489038089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6ca540 msgr2=0x56467b6db500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/489038089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6ca540 0x56467b6db500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c16d6640 1 -- 172.21.15.17:0/489038089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6c9b50 msgr2=0x56467b6d3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/489038089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6c9b50 0x56467b6d3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c16d6640 1 -- 172.21.15.17:0/489038089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56467b4096b0 con 0x56467b6cf6b0 2024-03-24T03:00:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.706+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/489038089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 0x56467b6d41b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd2b001f6c0 tx=0x7fd2b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddefa58cc90075e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.707+0000 7fd2a37fe640 1 -- 172.21.15.17:0/489038089 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b00053f0 con 0x56467b6cf6b0 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.707+0000 7fd2a37fe640 1 -- 172.21.15.17:0/489038089 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2b0025030 con 0x56467b6cf6b0 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.707+0000 7fd2a37fe640 1 -- 172.21.15.17:0/489038089 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b001fa60 con 0x56467b6cf6b0 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.707+0000 7fd2c6e00740 1 -- 172.21.15.17:0/489038089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 msgr2=0x56467b6d41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.707+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/489038089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 0x56467b6d41b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd2b001f6c0 tx=0x7fd2b0000bc0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.707+0000 7fd2c6e00740 1 -- 172.21.15.17:0/489038089 shutdown_connections 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.707+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/489038089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6cf6b0 0x56467b6d41b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.707+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/489038089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6ca540 0x56467b6db500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.707+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/489038089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6c9b50 0x56467b6d3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.707+0000 7fd2c6e00740 1 -- 172.21.15.17:0/489038089 >> 172.21.15.17:0/489038089 conn(0x56467b68da60 msgr2=0x56467b6cb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.707+0000 7fd2c6e00740 1 -- 172.21.15.17:0/489038089 shutdown_connections 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.707+0000 7fd2c6e00740 1 -- 172.21.15.17:0/489038089 wait complete. 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c6e00740 1 -- start start 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c6e00740 1 Processor -- start 2024-03-24T03:00:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c6e00740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6c9b50 0x56467b6d7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c6e00740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6ca540 0x56467b6d7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c6e00740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6cf6b0 0x56467b6daa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c6e00740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467b6e1670 con 0x56467b6c9b50 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c6e00740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467b6e1010 con 0x56467b6cf6b0 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c6e00740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467b6e1cd0 con 0x56467b6ca540 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c6e00740 10 client.0 _ll_register_callbacks cb 0x7fd2ac0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c16d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6c9b50 0x56467b6d7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c16d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6c9b50 0x56467b6d7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41970/0 (socket says 172.21.15.17:41970) 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c16d6640 1 -- 172.21.15.17:0/56826129 learned_addr learned my addr 172.21.15.17:0/56826129 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c1ed7640 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6ca540 0x56467b6d7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6cf6b0 0x56467b6daa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c16d6640 1 -- 172.21.15.17:0/56826129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6ca540 msgr2=0x56467b6d7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6ca540 0x56467b6d7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c16d6640 1 -- 172.21.15.17:0/56826129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6cf6b0 msgr2=0x56467b6daa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.708+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6cf6b0 0x56467b6daa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2c16d6640 1 -- 172.21.15.17:0/56826129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56467b6db060 con 0x56467b6c9b50 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2c16d6640 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6c9b50 0x56467b6d7510 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009480 tx=0x7fd2b0009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a176cd56f76d5150 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2a17fa640 1 -- 172.21.15.17:0/56826129 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000f990 con 0x56467b6c9b50 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2a17fa640 1 -- 172.21.15.17:0/56826129 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fd2b000fb30 con 0x56467b6c9b50 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2c6e00740 1 -- 172.21.15.17:0/56826129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2b0025030 con 0x56467b6c9b50 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2a17fa640 1 -- 172.21.15.17:0/56826129 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2b0048050 con 0x56467b6c9b50 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2c6e00740 10 client.15471 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2c6e00740 1 -- 172.21.15.17:0/56826129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56467b73bcd0 con 0x56467b6c9b50 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2a17fa640 1 -- 172.21.15.17:0/56826129 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2b0025030 con 0x56467b6c9b50 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2a17fa640 1 -- 172.21.15.17:0/56826129 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd2b0043030 con 0x56467b6c9b50 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2c6e00740 10 client.15471 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2c6e00740 10 client.15471 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2c6e00740 20 client.15471 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x56467b73ff50 0x56467b73bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2c6e00740 4 client.15471 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.709+0000 7fd2c6e00740 1 -- 172.21.15.17:0/56826129 --> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56467b6324b0 con 0x56467b73ff50 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c1ed7640 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x56467b73ff50 0x56467b73bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2a17fa640 10 client.15471 ms_handle_connect on v2:172.21.15.188:6834/3649219818 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c1ed7640 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x56467b73ff50 0x56467b73bea0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cf2a176b6c2a2c90 server_cookie=cb11d572d364ab62 in_seq=0 out_seq=0 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2a17fa640 1 -- 172.21.15.17:0/56826129 <== mds.3 v2:172.21.15.188:6834/3649219818 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56467b6324b0 con 0x56467b73ff50 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2a17fa640 10 client.15471 handle_command_reply: tid=0 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c6e00740 1 client.15471 shutdown 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c6e00740 1 -- 172.21.15.17:0/56826129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6c9b50 msgr2=0x56467b6d7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6c9b50 0x56467b6d7510 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009480 tx=0x7fd2b0009fb0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c6e00740 1 -- 172.21.15.17:0/56826129 shutdown_connections 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.188:6834/3649219818,v1:172.21.15.188:6835/3649219818] conn(0x56467b73ff50 0x56467b73bea0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56467b6cf6b0 0x56467b6daa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2a17fa640 0 client.15471 ms_handle_reset on v2:172.21.15.188:6834/3649219818 2024-03-24T03:00:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56467b6ca540 0x56467b6d7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c6e00740 1 --2- 172.21.15.17:0/56826129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56467b6c9b50 0x56467b6d7510 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c6e00740 1 -- 172.21.15.17:0/56826129 >> 172.21.15.17:0/56826129 conn(0x56467b68da60 msgr2=0x56467b6d3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c6e00740 1 -- 172.21.15.17:0/56826129 shutdown_connections 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.710+0000 7fd2c6e00740 1 -- 172.21.15.17:0/56826129 wait complete. 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c6e00740 20 client.15471 trim_cache size 0 max 16384 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 3, 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24493, 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-24T03:00:49.717 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 112.65454582700001, 2024-03-24T03:00:49.718 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 28, 2024-03-24T03:00:49.718 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 93, 2024-03-24T03:00:49.718 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-03-24T03:00:49.718 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 155.95815067800001 2024-03-24T03:00:49.718 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:00:49.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c413b640 1 -- 172.21.15.17:0/113085155 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd2a40a2800 msgr2=0x7fd2a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c413b640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd2a40a2800 0x7fd2a40a4cc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0984b0 tx=0x7fd2ac01bb30 comp rx=0 tx=0).stop 2024-03-24T03:00:49.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c413b640 1 -- 172.21.15.17:0/113085155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2bc103270 msgr2=0x7fd2bc19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c413b640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2bc103270 0x7fd2bc19ef80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00eb10 tx=0x7fd2ac00c2d0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c413b640 1 -- 172.21.15.17:0/113085155 shutdown_connections 2024-03-24T03:00:49.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c413b640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fd2a40a2800 0x7fd2a40a4cc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c413b640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2bc103270 0x7fd2bc19ef80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c413b640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2bc1028b0 0x7fd2bc197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c413b640 1 --2- 172.21.15.17:0/113085155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2bc1016c0 0x7fd2bc1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c413b640 1 -- 172.21.15.17:0/113085155 >> 172.21.15.17:0/113085155 conn(0x7fd2bc0fcc30 msgr2=0x7fd2bc105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c413b640 1 -- 172.21.15.17:0/113085155 shutdown_connections 2024-03-24T03:00:49.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.711+0000 7fd2c413b640 1 -- 172.21.15.17:0/113085155 wait complete. 2024-03-24T03:00:49.722 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-24T03:00:49.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.825+0000 7fbb90e4b640 1 -- 172.21.15.17:0/235255764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb8c0fff90 msgr2=0x7fbb8c1003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.825+0000 7fbb90e4b640 1 --2- 172.21.15.17:0/235255764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb8c0fff90 0x7fbb8c1003b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009900 tx=0x7fbb7c02bcd0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.825+0000 7fbb90e4b640 1 -- 172.21.15.17:0/235255764 shutdown_connections 2024-03-24T03:00:49.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.825+0000 7fbb90e4b640 1 --2- 172.21.15.17:0/235255764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb8c101290 0x7fbb8c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.825+0000 7fbb90e4b640 1 --2- 172.21.15.17:0/235255764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c1008f0 0x7fbb8c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.825+0000 7fbb90e4b640 1 --2- 172.21.15.17:0/235255764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb8c0fff90 0x7fbb8c1003b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.825+0000 7fbb90e4b640 1 -- 172.21.15.17:0/235255764 >> 172.21.15.17:0/235255764 conn(0x7fbb8c0fbb30 msgr2=0x7fbb8c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.825+0000 7fbb90e4b640 1 -- 172.21.15.17:0/235255764 shutdown_connections 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.825+0000 7fbb90e4b640 1 -- 172.21.15.17:0/235255764 wait complete. 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb90e4b640 1 Processor -- start 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb90e4b640 1 -- start start 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb90e4b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb8c0fff90 0x7fbb8c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb90e4b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb8c1008f0 0x7fbb8c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb90e4b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c101290 0x7fbb8c1a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb90e4b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c10f4f0 con 0x7fbb8c0fff90 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb90e4b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c10f370 con 0x7fbb8c101290 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb90e4b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c10f670 con 0x7fbb8c1008f0 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb8a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb8c0fff90 0x7fbb8c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb8a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb8c0fff90 0x7fbb8c199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41988/0 (socket says 172.21.15.17:41988) 2024-03-24T03:00:49.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb8a59c640 1 -- 172.21.15.17:0/94060122 learned_addr learned my addr 172.21.15.17:0/94060122 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb89d9b640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb8c1008f0 0x7fbb8c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb8ad9d640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c101290 0x7fbb8c1a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb89d9b640 1 -- 172.21.15.17:0/94060122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c101290 msgr2=0x7fbb8c1a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb89d9b640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c101290 0x7fbb8c1a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb89d9b640 1 -- 172.21.15.17:0/94060122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb8c0fff90 msgr2=0x7fbb8c199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb89d9b640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb8c0fff90 0x7fbb8c199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb89d9b640 1 -- 172.21.15.17:0/94060122 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c1a1850 con 0x7fbb8c1008f0 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.826+0000 7fbb8a59c640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb8c0fff90 0x7fbb8c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.827+0000 7fbb89d9b640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb8c1008f0 0x7fbb8c19a180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000e9f0 tx=0x7fbb8000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.827+0000 7fbb7b7fe640 1 -- 172.21.15.17:0/94060122 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000ccc0 con 0x7fbb8c1008f0 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.827+0000 7fbb90e4b640 1 -- 172.21.15.17:0/94060122 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c1a1b40 con 0x7fbb8c1008f0 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.827+0000 7fbb7b7fe640 1 -- 172.21.15.17:0/94060122 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb800035f0 con 0x7fbb8c1008f0 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.827+0000 7fbb90e4b640 1 -- 172.21.15.17:0/94060122 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c1a20e0 con 0x7fbb8c1008f0 2024-03-24T03:00:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.827+0000 7fbb7b7fe640 1 -- 172.21.15.17:0/94060122 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000ce60 con 0x7fbb8c1008f0 2024-03-24T03:00:49.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.829+0000 7fbb93a39740 1 Processor -- start 2024-03-24T03:00:49.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.829+0000 7fbb7b7fe640 1 -- 172.21.15.17:0/94060122 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbb800037a0 con 0x7fbb8c1008f0 2024-03-24T03:00:49.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.829+0000 7fbb93a39740 1 -- start start 2024-03-24T03:00:49.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb93a39740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087501740 0x55c087501b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb93a39740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 0x55c087506c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb93a39740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0875072a0 0x55c087509730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb93a39740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c087466c20 con 0x55c087502130 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb93a39740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08740fa40 con 0x55c0875072a0 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb93a39740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c087462cc0 con 0x55c087501740 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb8a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 0x55c087506c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb8a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 0x55c087506c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42000/0 (socket says 172.21.15.17:42000) 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb8a59c640 1 -- 172.21.15.17:0/2068492788 learned_addr learned my addr 172.21.15.17:0/2068492788 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb89d9b640 1 --2- 172.21.15.17:0/2068492788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0875072a0 0x55c087509730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb8ad9d640 1 --2- 172.21.15.17:0/2068492788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087501740 0x55c087501b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb7b7fe640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbb740a2800 0x7fbb740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb8a59c640 1 -- 172.21.15.17:0/2068492788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087501740 msgr2=0x55c087501b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb8a59c640 1 --2- 172.21.15.17:0/2068492788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087501740 0x55c087501b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb8a59c640 1 -- 172.21.15.17:0/2068492788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0875072a0 msgr2=0x55c087509730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb8a59c640 1 --2- 172.21.15.17:0/2068492788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0875072a0 0x55c087509730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb8a59c640 1 -- 172.21.15.17:0/2068492788 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c087431f90 con 0x55c087502130 2024-03-24T03:00:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb8ad9d640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbb740a2800 0x7fbb740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb7b7fe640 1 -- 172.21.15.17:0/94060122 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fbb800cf180 con 0x7fbb8c1008f0 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb8a59c640 1 --2- 172.21.15.17:0/2068492788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 0x55c087506c30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009230 tx=0x7fbb7c02cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbd295618d7b650 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.830+0000 7fbb78ff9640 1 -- 172.21.15.17:0/2068492788 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c042070 con 0x55c087502130 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.831+0000 7fbb78ff9640 1 -- 172.21.15.17:0/2068492788 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb7c004ba0 con 0x55c087502130 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.831+0000 7fbb78ff9640 1 -- 172.21.15.17:0/2068492788 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0052f0 con 0x55c087502130 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.831+0000 7fbb93a39740 1 -- 172.21.15.17:0/2068492788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 msgr2=0x55c087506c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.831+0000 7fbb8ad9d640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbb740a2800 0x7fbb740a4cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55c087502990 tx=0x7fbb84004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.831+0000 7fbb93a39740 1 --2- 172.21.15.17:0/2068492788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 0x55c087506c30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009230 tx=0x7fbb7c02cdd0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.831+0000 7fbb93a39740 1 -- 172.21.15.17:0/2068492788 shutdown_connections 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.831+0000 7fbb93a39740 1 --2- 172.21.15.17:0/2068492788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0875072a0 0x55c087509730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.831+0000 7fbb93a39740 1 --2- 172.21.15.17:0/2068492788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 0x55c087506c30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.831+0000 7fbb93a39740 1 --2- 172.21.15.17:0/2068492788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087501740 0x55c087501b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.831+0000 7fbb93a39740 1 -- 172.21.15.17:0/2068492788 >> 172.21.15.17:0/2068492788 conn(0x55c0874c9a60 msgr2=0x55c0874fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.831+0000 7fbb93a39740 1 -- 172.21.15.17:0/2068492788 shutdown_connections 2024-03-24T03:00:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.831+0000 7fbb93a39740 1 -- 172.21.15.17:0/2068492788 wait complete. 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb93a39740 1 -- start start 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb93a39740 1 Processor -- start 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb93a39740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087501740 0x55c08750bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb93a39740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 0x55c08750c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb93a39740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0875072a0 0x55c0875131a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb93a39740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08750a1f0 con 0x55c087502130 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb93a39740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c087422e90 con 0x55c087501740 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb93a39740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08750a370 con 0x55c0875072a0 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb93a39740 10 client.0 _ll_register_callbacks cb 0x55c08717aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb89d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 0x55c08750c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb89d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 0x55c08750c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42002/0 (socket says 172.21.15.17:42002) 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb89d9b640 1 -- 172.21.15.17:0/527433384 learned_addr learned my addr 172.21.15.17:0/527433384 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb8a59c640 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087501740 0x55c08750bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb8ad9d640 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0875072a0 0x55c0875131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb89d9b640 1 -- 172.21.15.17:0/527433384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0875072a0 msgr2=0x55c0875131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb89d9b640 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0875072a0 0x55c0875131a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb89d9b640 1 -- 172.21.15.17:0/527433384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087501740 msgr2=0x55c08750bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb89d9b640 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087501740 0x55c08750bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb89d9b640 1 -- 172.21.15.17:0/527433384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c08742f9d0 con 0x55c087502130 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb8ad9d640 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0875072a0 0x55c0875131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb89d9b640 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 0x55c08750c160 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbb800a3950 tx=0x7fbb80009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54eb185cd96dff96 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.832+0000 7fbb71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb71ffb640 1 -- 172.21.15.17:0/527433384 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8009b220 con 0x55c087502130 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb71ffb640 1 -- 172.21.15.17:0/527433384 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fbb800d2db0 con 0x55c087502130 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb93a39740 1 -- 172.21.15.17:0/527433384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb7c004ba0 con 0x55c087502130 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb71ffb640 1 -- 172.21.15.17:0/527433384 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb80099820 con 0x55c087502130 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb93a39740 10 client.15483 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb93a39740 1 -- 172.21.15.17:0/527433384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c08717b2d0 con 0x55c087502130 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb71ffb640 1 -- 172.21.15.17:0/527433384 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb7c004ba0 con 0x55c087502130 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb71ffb640 1 -- 172.21.15.17:0/527433384 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbb80084be0 con 0x55c087502130 2024-03-24T03:00:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb93a39740 10 client.15483 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb93a39740 10 client.15483 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb93a39740 20 client.15483 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb93a39740 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x55c087518480 0x55c087517530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb93a39740 4 client.15483 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb93a39740 1 -- 172.21.15.17:0/527433384 --> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c087474160 con 0x55c087518480 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.833+0000 7fbb8a59c640 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x55c087518480 0x55c087517530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.834+0000 7fbb8a59c640 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x55c087518480 0x55c087517530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d795bc93f0cbac68 server_cookie=a7475a83ddf42d49 in_seq=0 out_seq=0 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.834+0000 7fbb71ffb640 10 client.15483 ms_handle_connect on v2:172.21.15.188:6836/3512350624 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb71ffb640 1 -- 172.21.15.17:0/527433384 <== mds.2 v2:172.21.15.188:6836/3512350624 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c087474160 con 0x55c087518480 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb71ffb640 10 client.15483 handle_command_reply: tid=0 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb93a39740 1 client.15483 shutdown 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb93a39740 1 -- 172.21.15.17:0/527433384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 msgr2=0x55c08750c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb93a39740 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 0x55c08750c160 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbb800a3950 tx=0x7fbb80009490 comp rx=0 tx=0).stop 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb93a39740 1 -- 172.21.15.17:0/527433384 shutdown_connections 2024-03-24T03:00:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb93a39740 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c0875072a0 0x55c0875131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb93a39740 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c087502130 0x55c08750c160 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb93a39740 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x55c087518480 0x55c087517530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb93a39740 1 --2- 172.21.15.17:0/527433384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087501740 0x55c08750bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb71ffb640 0 client.15483 ms_handle_reset on v2:172.21.15.188:6836/3512350624 2024-03-24T03:00:49.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb93a39740 1 -- 172.21.15.17:0/527433384 >> 172.21.15.17:0/527433384 conn(0x55c0874c9a60 msgr2=0x55c0874fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb93a39740 1 -- 172.21.15.17:0/527433384 shutdown_connections 2024-03-24T03:00:49.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb93a39740 1 -- 172.21.15.17:0/527433384 wait complete. 2024-03-24T03:00:49.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.835+0000 7fbb93a39740 20 client.15483 trim_cache size 0 max 16384 2024-03-24T03:00:49.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb93a39740 1 Processor -- start 2024-03-24T03:00:49.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb93a39740 1 -- start start 2024-03-24T03:00:49.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb93a39740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087501740 0x55c087503670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb93a39740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087502130 0x55c087503bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb93a39740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 0x55c087513160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb93a39740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c087517ca0 con 0x55c0875072a0 2024-03-24T03:00:49.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb93a39740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c087518300 con 0x55c087501740 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb93a39740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c087518960 con 0x55c087502130 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb8ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 0x55c087513160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb8a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087501740 0x55c087503670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb8ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 0x55c087513160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42016/0 (socket says 172.21.15.17:42016) 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb8ad9d640 1 -- 172.21.15.17:0/252816117 learned_addr learned my addr 172.21.15.17:0/252816117 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb89d9b640 1 --2- 172.21.15.17:0/252816117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087502130 0x55c087503bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb8ad9d640 1 -- 172.21.15.17:0/252816117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087502130 msgr2=0x55c087503bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb8ad9d640 1 --2- 172.21.15.17:0/252816117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087502130 0x55c087503bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb8ad9d640 1 -- 172.21.15.17:0/252816117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087501740 msgr2=0x55c087503670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb8ad9d640 1 --2- 172.21.15.17:0/252816117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087501740 0x55c087503670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.852+0000 7fbb8ad9d640 1 -- 172.21.15.17:0/252816117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0872456b0 con 0x55c0875072a0 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.853+0000 7fbb8ad9d640 1 --2- 172.21.15.17:0/252816117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 0x55c087513160 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400f6c0 tx=0x7fbb84013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3e6986380c1f25 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.853+0000 7fbb73fff640 1 -- 172.21.15.17:0/252816117 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84004b20 con 0x55c0875072a0 2024-03-24T03:00:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.853+0000 7fbb73fff640 1 -- 172.21.15.17:0/252816117 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb84004cc0 con 0x55c0875072a0 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.853+0000 7fbb73fff640 1 -- 172.21.15.17:0/252816117 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84005360 con 0x55c0875072a0 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.853+0000 7fbb93a39740 1 -- 172.21.15.17:0/252816117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 msgr2=0x55c087513160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.853+0000 7fbb93a39740 1 --2- 172.21.15.17:0/252816117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 0x55c087513160 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400f6c0 tx=0x7fbb84013030 comp rx=0 tx=0).stop 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.853+0000 7fbb93a39740 1 -- 172.21.15.17:0/252816117 shutdown_connections 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.853+0000 7fbb93a39740 1 --2- 172.21.15.17:0/252816117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 0x55c087513160 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.853+0000 7fbb93a39740 1 --2- 172.21.15.17:0/252816117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087502130 0x55c087503bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.853+0000 7fbb93a39740 1 --2- 172.21.15.17:0/252816117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087501740 0x55c087503670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.853+0000 7fbb93a39740 1 -- 172.21.15.17:0/252816117 >> 172.21.15.17:0/252816117 conn(0x55c0874c9a60 msgr2=0x55c0874fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.853+0000 7fbb93a39740 1 -- 172.21.15.17:0/252816117 shutdown_connections 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb93a39740 1 -- 172.21.15.17:0/252816117 wait complete. 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb93a39740 1 -- start start 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb93a39740 1 Processor -- start 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb93a39740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087501740 0x55c08750f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb93a39740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087502130 0x55c08750f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb93a39740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 0x55c087512890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb93a39740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c087519620 con 0x55c0875072a0 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb93a39740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c087518fc0 con 0x55c087502130 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb93a39740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c087519c80 con 0x55c087501740 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb93a39740 10 client.0 _ll_register_callbacks cb 0x7fbb8000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb8ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 0x55c087512890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb8ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 0x55c087512890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42020/0 (socket says 172.21.15.17:42020) 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb8ad9d640 1 -- 172.21.15.17:0/3836000432 learned_addr learned my addr 172.21.15.17:0/3836000432 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb8a59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087501740 0x55c08750f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb89d9b640 1 --2- 172.21.15.17:0/3836000432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087502130 0x55c08750f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb8ad9d640 1 -- 172.21.15.17:0/3836000432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087501740 msgr2=0x55c08750f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb8ad9d640 1 --2- 172.21.15.17:0/3836000432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087501740 0x55c08750f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb8ad9d640 1 -- 172.21.15.17:0/3836000432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087502130 msgr2=0x55c08750f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb8ad9d640 1 --2- 172.21.15.17:0/3836000432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087502130 0x55c08750f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.854+0000 7fbb8ad9d640 1 -- 172.21.15.17:0/3836000432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c087512e80 con 0x55c0875072a0 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb8ad9d640 1 --2- 172.21.15.17:0/3836000432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 0x55c087512890 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbb84006010 tx=0x7fbb8401df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb394850eeaa7db server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb71ffb640 1 -- 172.21.15.17:0/3836000432 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84035030 con 0x55c0875072a0 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb71ffb640 1 -- 172.21.15.17:0/3836000432 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fbb8402d070 con 0x55c0875072a0 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb93a39740 1 -- 172.21.15.17:0/3836000432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb84004cc0 con 0x55c0875072a0 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb71ffb640 1 -- 172.21.15.17:0/3836000432 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb84042050 con 0x55c0875072a0 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb93a39740 10 client.15489 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb93a39740 1 -- 172.21.15.17:0/3836000432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c087573fc0 con 0x55c0875072a0 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb71ffb640 1 -- 172.21.15.17:0/3836000432 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb84004cc0 con 0x55c0875072a0 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb71ffb640 1 -- 172.21.15.17:0/3836000432 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbb84031440 con 0x55c0875072a0 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb93a39740 10 client.15489 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb93a39740 10 client.15489 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb93a39740 20 client.15489 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb93a39740 1 --2- 172.21.15.17:0/3836000432 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x55c087578400 0x55c087574190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb93a39740 4 client.15489 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb93a39740 1 -- 172.21.15.17:0/3836000432 --> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c08746e4b0 con 0x55c087578400 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.855+0000 7fbb8a59c640 1 --2- 172.21.15.17:0/3836000432 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x55c087578400 0x55c087574190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb71ffb640 10 client.15489 ms_handle_connect on v2:172.21.15.188:6836/3512350624 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb8a59c640 1 --2- 172.21.15.17:0/3836000432 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x55c087578400 0x55c087574190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9b633c1eb2b0e5b server_cookie=8190d4a490aff57f in_seq=0 out_seq=0 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb71ffb640 1 -- 172.21.15.17:0/3836000432 <== mds.2 v2:172.21.15.188:6836/3512350624 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55c08746e4b0 con 0x55c087578400 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb71ffb640 10 client.15489 handle_command_reply: tid=0 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb93a39740 1 client.15489 shutdown 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb93a39740 1 -- 172.21.15.17:0/3836000432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 msgr2=0x55c087512890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb93a39740 1 --2- 172.21.15.17:0/3836000432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 0x55c087512890 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbb84006010 tx=0x7fbb8401df70 comp rx=0 tx=0).stop 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb93a39740 1 -- 172.21.15.17:0/3836000432 shutdown_connections 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb93a39740 1 --2- 172.21.15.17:0/3836000432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c0875072a0 0x55c087512890 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb93a39740 1 --2- 172.21.15.17:0/3836000432 >> [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624] conn(0x55c087578400 0x55c087574190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb93a39740 1 --2- 172.21.15.17:0/3836000432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c087502130 0x55c08750f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb93a39740 1 --2- 172.21.15.17:0/3836000432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c087501740 0x55c08750f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb71ffb640 0 client.15489 ms_handle_reset on v2:172.21.15.188:6836/3512350624 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb93a39740 1 -- 172.21.15.17:0/3836000432 >> 172.21.15.17:0/3836000432 conn(0x55c0874c9a60 msgr2=0x55c08750bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb93a39740 1 -- 172.21.15.17:0/3836000432 shutdown_connections 2024-03-24T03:00:49.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb93a39740 1 -- 172.21.15.17:0/3836000432 wait complete. 2024-03-24T03:00:49.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.856+0000 7fbb93a39740 20 client.15489 trim_cache size 0 max 16384 2024-03-24T03:00:49.863 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:00:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T03:00:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 2, 2024-03-24T03:00:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24511, 2024-03-24T03:00:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-24T03:00:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-24T03:00:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-24T03:00:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 113.80854526500001, 2024-03-24T03:00:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 28, 2024-03-24T03:00:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 93, 2024-03-24T03:00:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-03-24T03:00:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 153.221647709 2024-03-24T03:00:49.866 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:00:49.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.857+0000 7fbb90e4b640 1 -- 172.21.15.17:0/94060122 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbb740a2800 msgr2=0x7fbb740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.857+0000 7fbb90e4b640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbb740a2800 0x7fbb740a4cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55c087502990 tx=0x7fbb84004a40 comp rx=0 tx=0).stop 2024-03-24T03:00:49.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.857+0000 7fbb90e4b640 1 -- 172.21.15.17:0/94060122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb8c1008f0 msgr2=0x7fbb8c19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.857+0000 7fbb90e4b640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb8c1008f0 0x7fbb8c19a180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000e9f0 tx=0x7fbb8000edf0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.857+0000 7fbb90e4b640 1 -- 172.21.15.17:0/94060122 shutdown_connections 2024-03-24T03:00:49.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.857+0000 7fbb90e4b640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fbb740a2800 0x7fbb740a4cc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.857+0000 7fbb90e4b640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c101290 0x7fbb8c1a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.857+0000 7fbb90e4b640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb8c1008f0 0x7fbb8c19a180 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.857+0000 7fbb90e4b640 1 --2- 172.21.15.17:0/94060122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb8c0fff90 0x7fbb8c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.857+0000 7fbb90e4b640 1 -- 172.21.15.17:0/94060122 >> 172.21.15.17:0/94060122 conn(0x7fbb8c0fbb30 msgr2=0x7fbb8c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.857+0000 7fbb90e4b640 1 -- 172.21.15.17:0/94060122 shutdown_connections 2024-03-24T03:00:49.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.857+0000 7fbb90e4b640 1 -- 172.21.15.17:0/94060122 wait complete. 2024-03-24T03:00:49.868 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-24T03:00:49.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.973+0000 7f5997d1d640 1 -- 172.21.15.17:0/1598002841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59901016b0 msgr2=0x7f5990101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.973+0000 7f5997d1d640 1 --2- 172.21.15.17:0/1598002841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59901016b0 0x7f5990101ad0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f598c009900 tx=0x7f598c02bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:49.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.973+0000 7f5997d1d640 1 -- 172.21.15.17:0/1598002841 shutdown_connections 2024-03-24T03:00:49.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.973+0000 7f5997d1d640 1 --2- 172.21.15.17:0/1598002841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5990103260 0x7f5990109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.973+0000 7f5997d1d640 1 --2- 172.21.15.17:0/1598002841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59901028a0 0x7f5990102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.973+0000 7f5997d1d640 1 --2- 172.21.15.17:0/1598002841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59901016b0 0x7f5990101ad0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.973+0000 7f5997d1d640 1 -- 172.21.15.17:0/1598002841 >> 172.21.15.17:0/1598002841 conn(0x7f59900fcc40 msgr2=0x7f59900ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.973+0000 7f5997d1d640 1 -- 172.21.15.17:0/1598002841 shutdown_connections 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.973+0000 7f5997d1d640 1 -- 172.21.15.17:0/1598002841 wait complete. 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.973+0000 7f5997d1d640 1 Processor -- start 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.973+0000 7f5997d1d640 1 -- start start 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f5997d1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59901016b0 0x7f5990197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f5997d1d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59901028a0 0x7f5990197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f5997d1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5990103260 0x7f599019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f5997d1d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599010d330 con 0x7f59901028a0 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f5997d1d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599010d1b0 con 0x7f5990103260 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f5997d1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599010d4b0 con 0x7f59901016b0 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f59952b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59901028a0 0x7f5990197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f59952b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59901028a0 0x7f5990197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42028/0 (socket says 172.21.15.17:42028) 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f59952b8640 1 -- 172.21.15.17:0/18793760 learned_addr learned my addr 172.21.15.17:0/18793760 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f5995ab9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59901016b0 0x7f5990197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f59962ba640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5990103260 0x7f599019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f59952b8640 1 -- 172.21.15.17:0/18793760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59901016b0 msgr2=0x7f5990197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f59952b8640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59901016b0 0x7f5990197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f59952b8640 1 -- 172.21.15.17:0/18793760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5990103260 msgr2=0x7f599019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f59952b8640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5990103260 0x7f599019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f59952b8640 1 -- 172.21.15.17:0/18793760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599019f650 con 0x7f59901028a0 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f59952b8640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59901028a0 0x7f5990197f70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f598000e9f0 tx=0x7f598000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f5986ffd640 1 -- 172.21.15.17:0/18793760 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598000cd80 con 0x7f59901028a0 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f5986ffd640 1 -- 172.21.15.17:0/18793760 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5980005500 con 0x7f59901028a0 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f5997d1d640 1 -- 172.21.15.17:0/18793760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599019f940 con 0x7f59901028a0 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f5986ffd640 1 -- 172.21.15.17:0/18793760 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980010690 con 0x7f59901028a0 2024-03-24T03:00:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.974+0000 7f5997d1d640 1 -- 172.21.15.17:0/18793760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599019fe40 con 0x7f59901028a0 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.976+0000 7f5986ffd640 1 -- 172.21.15.17:0/18793760 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f59800046c0 con 0x7f59901028a0 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.977+0000 7f599a9df740 1 Processor -- start 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.977+0000 7f599a9df740 1 -- start start 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.977+0000 7f599a9df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a9673b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.977+0000 7f599a9df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 0x5590a9678c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.977+0000 7f599a9df740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a967b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.977+0000 7f599a9df740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a95d8c20 con 0x5590a96792a0 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.977+0000 7f599a9df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a9581a40 con 0x5590a9673740 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.977+0000 7f599a9df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a95d4cc0 con 0x5590a9674130 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.977+0000 7f5995ab9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 0x5590a9678c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f59952b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a967b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f5995ab9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 0x5590a9678c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:43108/0 (socket says 172.21.15.17:43108) 2024-03-24T03:00:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f5995ab9640 1 -- 172.21.15.17:0/1907256523 learned_addr learned my addr 172.21.15.17:0/1907256523 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f5986ffd640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f597c0a2800 0x7f597c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f59962ba640 1 --2- 172.21.15.17:0/1907256523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a9673b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f59962ba640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f597c0a2800 0x7f597c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f5995ab9640 1 -- 172.21.15.17:0/1907256523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 msgr2=0x5590a9673b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f5995ab9640 1 --2- 172.21.15.17:0/1907256523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a9673b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f5995ab9640 1 -- 172.21.15.17:0/1907256523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 msgr2=0x5590a967b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f5995ab9640 1 --2- 172.21.15.17:0/1907256523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a967b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f5995ab9640 1 -- 172.21.15.17:0/1907256523 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a95a3f90 con 0x5590a9674130 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f5986ffd640 1 -- 172.21.15.17:0/18793760 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f59800cf280 con 0x7f59901028a0 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f5995ab9640 1 --2- 172.21.15.17:0/1907256523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 0x5590a9678c30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f598c009900 tx=0x7f598c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a1027c4ed38813 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f59962ba640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f597c0a2800 0x7f597c0a4cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x5590a9674990 tx=0x7f5988004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f597bfff640 1 -- 172.21.15.17:0/1907256523 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c02cb80 con 0x5590a9674130 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f597bfff640 1 -- 172.21.15.17:0/1907256523 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f598c005040 con 0x5590a9674130 2024-03-24T03:00:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.978+0000 7f597bfff640 1 -- 172.21.15.17:0/1907256523 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c004920 con 0x5590a9674130 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.979+0000 7f599a9df740 1 -- 172.21.15.17:0/1907256523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 msgr2=0x5590a9678c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.979+0000 7f599a9df740 1 --2- 172.21.15.17:0/1907256523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 0x5590a9678c30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f598c009900 tx=0x7f598c009560 comp rx=0 tx=0).stop 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.979+0000 7f599a9df740 1 -- 172.21.15.17:0/1907256523 shutdown_connections 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.979+0000 7f599a9df740 1 --2- 172.21.15.17:0/1907256523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a967b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.979+0000 7f599a9df740 1 --2- 172.21.15.17:0/1907256523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 0x5590a9678c30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.979+0000 7f599a9df740 1 --2- 172.21.15.17:0/1907256523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a9673b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.979+0000 7f599a9df740 1 -- 172.21.15.17:0/1907256523 >> 172.21.15.17:0/1907256523 conn(0x5590a963ba60 msgr2=0x5590a966f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.979+0000 7f599a9df740 1 -- 172.21.15.17:0/1907256523 shutdown_connections 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.979+0000 7f599a9df740 1 -- 172.21.15.17:0/1907256523 wait complete. 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.979+0000 7f599a9df740 1 -- start start 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f599a9df740 1 Processor -- start 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f599a9df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a967dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f599a9df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 0x5590a967e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f599a9df740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a96851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f599a9df740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a967c1f0 con 0x5590a96792a0 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f599a9df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a9594e90 con 0x5590a9673740 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f599a9df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a967c370 con 0x5590a9674130 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f599a9df740 10 client.0 _ll_register_callbacks cb 0x5590a92ecac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f59962ba640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a96851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f59952b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 0x5590a967e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f5995ab9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a967dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f59962ba640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a96851c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42044/0 (socket says 172.21.15.17:42044) 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f59962ba640 1 -- 172.21.15.17:0/602920100 learned_addr learned my addr 172.21.15.17:0/602920100 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.980+0000 7f59962ba640 1 -- 172.21.15.17:0/602920100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 msgr2=0x5590a967e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f59962ba640 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 0x5590a967e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f59962ba640 1 -- 172.21.15.17:0/602920100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 msgr2=0x5590a967dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f59962ba640 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a967dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f59962ba640 1 -- 172.21.15.17:0/602920100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590a95a19d0 con 0x5590a96792a0 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f59952b8640 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 0x5590a967e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f5995ab9640 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a967dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f59962ba640 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a96851c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5988003dc0 tx=0x7f5988004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22eebedb44ade36f server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f59797fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f59797fa640 1 -- 172.21.15.17:0/602920100 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988025030 con 0x5590a96792a0 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f59797fa640 1 -- 172.21.15.17:0/602920100 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f5988004550 con 0x5590a96792a0 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f599a9df740 1 -- 172.21.15.17:0/602920100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f598c005040 con 0x5590a96792a0 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f59797fa640 1 -- 172.21.15.17:0/602920100 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5988031050 con 0x5590a96792a0 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f599a9df740 10 client.15501 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f599a9df740 1 -- 172.21.15.17:0/602920100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5590a92ed2d0 con 0x5590a96792a0 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f59797fa640 1 -- 172.21.15.17:0/602920100 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f598c005040 con 0x5590a96792a0 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f59797fa640 1 -- 172.21.15.17:0/602920100 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f598801f040 con 0x5590a96792a0 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f599a9df740 10 client.15501 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f599a9df740 10 client.15501 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f599a9df740 20 client.15501 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f599a9df740 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x5590a968a4a0 0x5590a96894b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f599a9df740 4 client.15501 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.981+0000 7f599a9df740 1 -- 172.21.15.17:0/602920100 --> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590a95e6160 con 0x5590a968a4a0 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.982+0000 7f5995ab9640 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x5590a968a4a0 0x5590a96894b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.982+0000 7f5995ab9640 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x5590a968a4a0 0x5590a96894b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc3fa19c3a1a7a4 server_cookie=c9038d1eec4106a4 in_seq=0 out_seq=0 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.982+0000 7f59797fa640 10 client.15501 ms_handle_connect on v2:172.21.15.19:6838/2102503456 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f59797fa640 1 -- 172.21.15.17:0/602920100 <== mds.1 v2:172.21.15.19:6838/2102503456 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f598c0504b0 con 0x5590a968a4a0 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f59797fa640 10 client.15501 handle_command_reply: tid=0 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f599a9df740 1 client.15501 shutdown 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f599a9df740 1 -- 172.21.15.17:0/602920100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 msgr2=0x5590a96851c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f599a9df740 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a96851c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5988003dc0 tx=0x7f5988004a40 comp rx=0 tx=0).stop 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f599a9df740 1 -- 172.21.15.17:0/602920100 shutdown_connections 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f599a9df740 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x5590a968a4a0 0x5590a96894b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f599a9df740 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a96851c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f599a9df740 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9674130 0x5590a967e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f599a9df740 1 --2- 172.21.15.17:0/602920100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a967dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f59797fa640 0 client.15501 ms_handle_reset on v2:172.21.15.19:6838/2102503456 2024-03-24T03:00:49.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f599a9df740 1 -- 172.21.15.17:0/602920100 >> 172.21.15.17:0/602920100 conn(0x5590a963ba60 msgr2=0x5590a966e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:49.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f599a9df740 1 -- 172.21.15.17:0/602920100 shutdown_connections 2024-03-24T03:00:49.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.983+0000 7f599a9df740 1 -- 172.21.15.17:0/602920100 wait complete. 2024-03-24T03:00:49.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:49.984+0000 7f599a9df740 20 client.15501 trim_cache size 0 max 16384 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.000+0000 7f599a9df740 1 Processor -- start 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.000+0000 7f599a9df740 1 -- start start 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.000+0000 7f599a9df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9673740 0x5590a9692940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.000+0000 7f599a9df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9674130 0x5590a9692e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.000+0000 7f599a9df740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a96933c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.000+0000 7f599a9df740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a9689cc0 con 0x5590a96792a0 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.000+0000 7f599a9df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a968a320 con 0x5590a9674130 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.000+0000 7f599a9df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a968a980 con 0x5590a9673740 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.000+0000 7f59962ba640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a96933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.000+0000 7f59962ba640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a96933c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42058/0 (socket says 172.21.15.17:42058) 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.000+0000 7f59962ba640 1 -- 172.21.15.17:0/3714280064 learned_addr learned my addr 172.21.15.17:0/3714280064 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.001+0000 7f5995ab9640 1 --2- 172.21.15.17:0/3714280064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9673740 0x5590a9692940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.001+0000 7f59952b8640 1 --2- 172.21.15.17:0/3714280064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9674130 0x5590a9692e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.001+0000 7f59962ba640 1 -- 172.21.15.17:0/3714280064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9673740 msgr2=0x5590a9692940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.001+0000 7f59962ba640 1 --2- 172.21.15.17:0/3714280064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9673740 0x5590a9692940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.001+0000 7f59962ba640 1 -- 172.21.15.17:0/3714280064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9674130 msgr2=0x5590a9692e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.001+0000 7f59962ba640 1 --2- 172.21.15.17:0/3714280064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9674130 0x5590a9692e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.001+0000 7f59962ba640 1 -- 172.21.15.17:0/3714280064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a93b76b0 con 0x5590a96792a0 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.001+0000 7f59962ba640 1 --2- 172.21.15.17:0/3714280064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a96933c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5988005a70 tx=0x7f5988005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9da8c7eef155bb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.001+0000 7f597b7fe640 1 -- 172.21.15.17:0/3714280064 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988025040 con 0x5590a96792a0 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.001+0000 7f597b7fe640 1 -- 172.21.15.17:0/3714280064 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5988004620 con 0x5590a96792a0 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.001+0000 7f597b7fe640 1 -- 172.21.15.17:0/3714280064 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988021d90 con 0x5590a96792a0 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 -- 172.21.15.17:0/3714280064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 msgr2=0x5590a96933c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 --2- 172.21.15.17:0/3714280064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a96933c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5988005a70 tx=0x7f5988005aa0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 -- 172.21.15.17:0/3714280064 shutdown_connections 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 --2- 172.21.15.17:0/3714280064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a96792a0 0x5590a96933c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 --2- 172.21.15.17:0/3714280064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9674130 0x5590a9692e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 --2- 172.21.15.17:0/3714280064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a9673740 0x5590a9692940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 -- 172.21.15.17:0/3714280064 >> 172.21.15.17:0/3714280064 conn(0x5590a963ba60 msgr2=0x5590a9675670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 -- 172.21.15.17:0/3714280064 shutdown_connections 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 -- 172.21.15.17:0/3714280064 wait complete. 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 -- start start 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 Processor -- start 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a96e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a9674130 0x5590a96e2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a96792a0 0x5590a96e8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a968b640 con 0x5590a9674130 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a968afe0 con 0x5590a9673740 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a968bca0 con 0x5590a96792a0 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.002+0000 7f599a9df740 10 client.0 _ll_register_callbacks cb 0x7f5988003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59952b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a9674130 0x5590a96e2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59962ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a96792a0 0x5590a96e8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59952b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a9674130 0x5590a96e2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42060/0 (socket says 172.21.15.17:42060) 2024-03-24T03:00:50.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59952b8640 1 -- 172.21.15.17:0/2559953810 learned_addr learned my addr 172.21.15.17:0/2559953810 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f5995ab9640 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a96e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59952b8640 1 -- 172.21.15.17:0/2559953810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a96792a0 msgr2=0x5590a96e8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59952b8640 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a96792a0 0x5590a96e8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59952b8640 1 -- 172.21.15.17:0/2559953810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 msgr2=0x5590a96e1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59952b8640 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a96e1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59952b8640 1 -- 172.21.15.17:0/2559953810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590a96e95b0 con 0x5590a9674130 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59962ba640 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a96792a0 0x5590a96e8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59952b8640 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a9674130 0x5590a96e2020 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f598009b770 tx=0x7f598009b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f85dd887d1141b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59797fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59797fa640 1 -- 172.21.15.17:0/2559953810 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598009bee0 con 0x5590a9674130 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59797fa640 1 -- 172.21.15.17:0/2559953810 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f5980097e50 con 0x5590a9674130 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f599a9df740 1 -- 172.21.15.17:0/2559953810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5988004620 con 0x5590a9674130 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59797fa640 1 -- 172.21.15.17:0/2559953810 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f598009f930 con 0x5590a9674130 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f599a9df740 10 client.15507 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f599a9df740 1 -- 172.21.15.17:0/2559953810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5590a96ea920 con 0x5590a9674130 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.003+0000 7f59797fa640 1 -- 172.21.15.17:0/2559953810 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5988004620 con 0x5590a9674130 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f59797fa640 1 -- 172.21.15.17:0/2559953810 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f59800afbc0 con 0x5590a9674130 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f599a9df740 10 client.15507 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f599a9df740 10 client.15507 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f599a9df740 20 client.15507 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f599a9df740 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x5590a96ef070 0x5590a96eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f599a9df740 4 client.15507 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f599a9df740 1 -- 172.21.15.17:0/2559953810 --> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5590a95e04b0 con 0x5590a96ef070 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f5995ab9640 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x5590a96ef070 0x5590a96eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f5995ab9640 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x5590a96ef070 0x5590a96eadd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13d0df919ccb7e6 server_cookie=8375efe159d9cc6d in_seq=0 out_seq=0 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f59797fa640 10 client.15507 ms_handle_connect on v2:172.21.15.19:6838/2102503456 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f59797fa640 1 -- 172.21.15.17:0/2559953810 <== mds.1 v2:172.21.15.19:6838/2102503456 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5590a95e04b0 con 0x5590a96ef070 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f59797fa640 10 client.15507 handle_command_reply: tid=0 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.004+0000 7f599a9df740 1 client.15507 shutdown 2024-03-24T03:00:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f599a9df740 1 -- 172.21.15.17:0/2559953810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a9674130 msgr2=0x5590a96e2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f599a9df740 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a9674130 0x5590a96e2020 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f598009b770 tx=0x7f598009b6e0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f599a9df740 1 -- 172.21.15.17:0/2559953810 shutdown_connections 2024-03-24T03:00:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f599a9df740 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.19:6838/2102503456,v1:172.21.15.19:6839/2102503456] conn(0x5590a96ef070 0x5590a96eadd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f599a9df740 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590a96792a0 0x5590a96e8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f599a9df740 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5590a9674130 0x5590a96e2020 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f599a9df740 1 --2- 172.21.15.17:0/2559953810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a9673740 0x5590a96e1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f59797fa640 0 client.15507 ms_handle_reset on v2:172.21.15.19:6838/2102503456 2024-03-24T03:00:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f599a9df740 1 -- 172.21.15.17:0/2559953810 >> 172.21.15.17:0/2559953810 conn(0x5590a963ba60 msgr2=0x5590a96722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f599a9df740 1 -- 172.21.15.17:0/2559953810 shutdown_connections 2024-03-24T03:00:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f599a9df740 1 -- 172.21.15.17:0/2559953810 wait complete. 2024-03-24T03:00:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f599a9df740 20 client.15507 trim_cache size 0 max 16384 2024-03-24T03:00:50.011 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:00:50.012 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T03:00:50.012 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 1, 2024-03-24T03:00:50.012 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24521, 2024-03-24T03:00:50.012 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-24T03:00:50.012 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-24T03:00:50.012 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-24T03:00:50.012 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 114.965616924, 2024-03-24T03:00:50.012 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 28, 2024-03-24T03:00:50.012 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 93, 2024-03-24T03:00:50.012 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-03-24T03:00:50.012 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 139.023508378 2024-03-24T03:00:50.012 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:00:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f5997d1d640 1 -- 172.21.15.17:0/18793760 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f597c0a2800 msgr2=0x7f597c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f5997d1d640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f597c0a2800 0x7f597c0a4cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x5590a9674990 tx=0x7f5988004ab0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f5997d1d640 1 -- 172.21.15.17:0/18793760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59901028a0 msgr2=0x7f5990197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f5997d1d640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59901028a0 0x7f5990197f70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f598000e9f0 tx=0x7f598000edf0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f5997d1d640 1 -- 172.21.15.17:0/18793760 shutdown_connections 2024-03-24T03:00:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f5997d1d640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f597c0a2800 0x7f597c0a4cc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f5997d1d640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5990103260 0x7f599019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f5997d1d640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59901028a0 0x7f5990197f70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f5997d1d640 1 --2- 172.21.15.17:0/18793760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59901016b0 0x7f5990197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f5997d1d640 1 -- 172.21.15.17:0/18793760 >> 172.21.15.17:0/18793760 conn(0x7f59900fcc40 msgr2=0x7f5990105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f5997d1d640 1 -- 172.21.15.17:0/18793760 shutdown_connections 2024-03-24T03:00:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.005+0000 7f5997d1d640 1 -- 172.21.15.17:0/18793760 wait complete. 2024-03-24T03:00:50.015 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:49 smithi017 ceph-mon[31926]: pgmap v362: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 55 B/s, 0 objects/s recovering 2024-03-24T03:00:50.015 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:49 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3270391664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:50.015 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:49 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1865167529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:50.016 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-24T03:00:50.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.119+0000 7f5756b77640 1 -- 172.21.15.17:0/3633325949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57501005b0 msgr2=0x7f57501009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.119+0000 7f5756b77640 1 --2- 172.21.15.17:0/3633325949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57501005b0 0x7f57501009d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5748009ca0 tx=0x7f574802c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.119+0000 7f5756b77640 1 -- 172.21.15.17:0/3633325949 shutdown_connections 2024-03-24T03:00:50.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.119+0000 7f5756b77640 1 --2- 172.21.15.17:0/3633325949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5750102160 0x7f57501089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.119+0000 7f5756b77640 1 --2- 172.21.15.17:0/3633325949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57501017a0 0x7f5750101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.119+0000 7f5756b77640 1 --2- 172.21.15.17:0/3633325949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57501005b0 0x7f57501009d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.119+0000 7f5756b77640 1 -- 172.21.15.17:0/3633325949 >> 172.21.15.17:0/3633325949 conn(0x7f57500fbb40 msgr2=0x7f57500fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:50.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.119+0000 7f5756b77640 1 -- 172.21.15.17:0/3633325949 shutdown_connections 2024-03-24T03:00:50.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.119+0000 7f5756b77640 1 -- 172.21.15.17:0/3633325949 wait complete. 2024-03-24T03:00:50.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.119+0000 7f5756b77640 1 Processor -- start 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.119+0000 7f5756b77640 1 -- start start 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5756b77640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57501005b0 0x7f575006a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5756b77640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57501017a0 0x7f575019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5756b77640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5750102160 0x7f575019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5756b77640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575010c200 con 0x7f57501017a0 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5756b77640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575010c080 con 0x7f57501005b0 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5756b77640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575010c380 con 0x7f5750102160 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f574ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57501017a0 0x7f575019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f574ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57501017a0 0x7f575019c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42080/0 (socket says 172.21.15.17:42080) 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f574ffff640 1 -- 172.21.15.17:0/620359258 learned_addr learned my addr 172.21.15.17:0/620359258 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5754913640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57501005b0 0x7f575006a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5755114640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5750102160 0x7f575019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5754913640 1 -- 172.21.15.17:0/620359258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5750102160 msgr2=0x7f575019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5754913640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5750102160 0x7f575019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5754913640 1 -- 172.21.15.17:0/620359258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57501017a0 msgr2=0x7f575019c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5754913640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57501017a0 0x7f575019c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5754913640 1 -- 172.21.15.17:0/620359258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575019f6b0 con 0x7f57501005b0 2024-03-24T03:00:50.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f574ffff640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57501017a0 0x7f575019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:50.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5754913640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57501005b0 0x7f575006a7e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f574802c630 tx=0x7f5748032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:50.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f574dffb640 1 -- 172.21.15.17:0/620359258 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748041020 con 0x7f57501005b0 2024-03-24T03:00:50.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f5756b77640 1 -- 172.21.15.17:0/620359258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f575019f8e0 con 0x7f57501005b0 2024-03-24T03:00:50.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f574dffb640 1 -- 172.21.15.17:0/620359258 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5748034cd0 con 0x7f57501005b0 2024-03-24T03:00:50.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.120+0000 7f574dffb640 1 -- 172.21.15.17:0/620359258 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748039040 con 0x7f57501005b0 2024-03-24T03:00:50.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.121+0000 7f5756b77640 1 -- 172.21.15.17:0/620359258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f575019fe20 con 0x7f57501005b0 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f574dffb640 1 -- 172.21.15.17:0/620359258 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5748044020 con 0x7f57501005b0 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f57593fd740 1 Processor -- start 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f57593fd740 1 -- start start 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f57593fd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294df740 0x55b3294dfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f57593fd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294e0130 0x55b3294e4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f57593fd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e52a0 0x55b3294e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f57593fd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b329444c20 con 0x55b3294e0130 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f57593fd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3293eda40 con 0x55b3294df740 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f57593fd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b329440cc0 con 0x55b3294e52a0 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f5755114640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294df740 0x55b3294dfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f5754913640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294e0130 0x55b3294e4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f5755114640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294df740 0x55b3294dfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:41434/0 (socket says 172.21.15.17:41434) 2024-03-24T03:00:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f574ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e52a0 0x55b3294e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f5755114640 1 -- 172.21.15.17:0/1834143995 learned_addr learned my addr 172.21.15.17:0/1834143995 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.123+0000 7f574dffb640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f57380a2800 0x7f57380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f5755114640 1 -- 172.21.15.17:0/1834143995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e52a0 msgr2=0x55b3294e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f5755114640 1 --2- 172.21.15.17:0/1834143995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e52a0 0x55b3294e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f5755114640 1 -- 172.21.15.17:0/1834143995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294e0130 msgr2=0x55b3294e4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f5755114640 1 --2- 172.21.15.17:0/1834143995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294e0130 0x55b3294e4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f5755114640 1 -- 172.21.15.17:0/1834143995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b32940ff90 con 0x55b3294df740 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f5754913640 1 --2- 172.21.15.17:0/1834143995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294e0130 0x55b3294e4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f574dffb640 1 -- 172.21.15.17:0/620359258 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f57480f0c10 con 0x7f57501005b0 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f5755114640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f57380a2800 0x7f57380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f5755114640 1 --2- 172.21.15.17:0/1834143995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294df740 0x55b3294dfb60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5740009d90 tx=0x7f5740013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a9879241612ca1d server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f5736ffd640 1 -- 172.21.15.17:0/1834143995 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57400037f0 con 0x55b3294df740 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f5736ffd640 1 -- 172.21.15.17:0/1834143995 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f574001f380 con 0x55b3294df740 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f5736ffd640 1 -- 172.21.15.17:0/1834143995 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574000d5a0 con 0x55b3294df740 2024-03-24T03:00:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.124+0000 7f5755114640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f57380a2800 0x7f57380a4cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5740027000 tx=0x7f5740024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.125+0000 7f57593fd740 1 -- 172.21.15.17:0/1834143995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294df740 msgr2=0x55b3294dfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.125+0000 7f57593fd740 1 --2- 172.21.15.17:0/1834143995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294df740 0x55b3294dfb60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5740009d90 tx=0x7f5740013ba0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.125+0000 7f57593fd740 1 -- 172.21.15.17:0/1834143995 shutdown_connections 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.125+0000 7f57593fd740 1 --2- 172.21.15.17:0/1834143995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e52a0 0x55b3294e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.125+0000 7f57593fd740 1 --2- 172.21.15.17:0/1834143995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294e0130 0x55b3294e4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.125+0000 7f57593fd740 1 --2- 172.21.15.17:0/1834143995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294df740 0x55b3294dfb60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.125+0000 7f57593fd740 1 -- 172.21.15.17:0/1834143995 >> 172.21.15.17:0/1834143995 conn(0x55b3294a7a60 msgr2=0x55b3294db0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.125+0000 7f57593fd740 1 -- 172.21.15.17:0/1834143995 shutdown_connections 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.125+0000 7f57593fd740 1 -- 172.21.15.17:0/1834143995 wait complete. 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.125+0000 7f57593fd740 1 -- start start 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.125+0000 7f57593fd740 1 Processor -- start 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f57593fd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294e9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f57593fd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294ea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f57593fd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f57593fd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3294e8250 con 0x55b3294df740 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f57593fd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b329400e90 con 0x55b3294e52a0 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f57593fd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3294e83d0 con 0x55b3294e0130 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f57593fd740 10 client.0 _ll_register_callbacks cb 0x55b329158ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f5754913640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294e9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f5754913640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294e9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42098/0 (socket says 172.21.15.17:42098) 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f5754913640 1 -- 172.21.15.17:0/10357652 learned_addr learned my addr 172.21.15.17:0/10357652 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f5755114640 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f574ffff640 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f5754913640 1 -- 172.21.15.17:0/10357652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 msgr2=0x55b3294ea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f5754913640 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294ea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f5754913640 1 -- 172.21.15.17:0/10357652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 msgr2=0x55b3294f11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f5754913640 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f5754913640 1 -- 172.21.15.17:0/10357652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b32940d9d0 con 0x55b3294df740 2024-03-24T03:00:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f5755114640 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f5754913640 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294e9c50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5748009ca0 tx=0x7f5748005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4935311964821c server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f572ffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f572ffff640 1 -- 172.21.15.17:0/10357652 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57480bcfd0 con 0x55b3294df740 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f572ffff640 1 -- 172.21.15.17:0/10357652 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f5748003970 con 0x55b3294df740 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f57593fd740 1 -- 172.21.15.17:0/10357652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f574001f380 con 0x55b3294df740 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f572ffff640 1 -- 172.21.15.17:0/10357652 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57480bcb60 con 0x55b3294df740 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f57593fd740 10 client.15519 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.126+0000 7f57593fd740 1 -- 172.21.15.17:0/10357652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b3291592d0 con 0x55b3294df740 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.127+0000 7f572ffff640 1 -- 172.21.15.17:0/10357652 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f574001f380 con 0x55b3294df740 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.127+0000 7f572ffff640 1 -- 172.21.15.17:0/10357652 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f574803c030 con 0x55b3294df740 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.127+0000 7f57593fd740 10 client.15519 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.127+0000 7f57593fd740 10 client.15519 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.127+0000 7f57593fd740 20 client.15519 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.127+0000 7f57593fd740 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55b3294f6490 0x55b3294f5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.127+0000 7f57593fd740 4 client.15519 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-24T03:00:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.127+0000 7f57593fd740 1 -- 172.21.15.17:0/10357652 --> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b329452160 con 0x55b3294f6490 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.127+0000 7f574ffff640 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55b3294f6490 0x55b3294f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.127+0000 7f572ffff640 10 client.15519 ms_handle_connect on v2:172.21.15.188:6838/81409664 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.127+0000 7f574ffff640 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55b3294f6490 0x55b3294f5330 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a36b4f80a6b67176 server_cookie=2857b8038c85a831 in_seq=0 out_seq=0 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f572ffff640 1 -- 172.21.15.17:0/10357652 <== mds.0 v2:172.21.15.188:6838/81409664 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b329452160 con 0x55b3294f6490 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f572ffff640 10 client.15519 handle_command_reply: tid=0 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f57593fd740 1 client.15519 shutdown 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f57593fd740 1 -- 172.21.15.17:0/10357652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 msgr2=0x55b3294e9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f57593fd740 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294e9c50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5748009ca0 tx=0x7f5748005ef0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f57593fd740 1 -- 172.21.15.17:0/10357652 shutdown_connections 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f57593fd740 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f57593fd740 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f57593fd740 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55b3294f6490 0x55b3294f5330 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f57593fd740 1 --2- 172.21.15.17:0/10357652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294e9c50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f572ffff640 0 client.15519 ms_handle_reset on v2:172.21.15.188:6838/81409664 2024-03-24T03:00:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f57593fd740 1 -- 172.21.15.17:0/10357652 >> 172.21.15.17:0/10357652 conn(0x55b3294a7a60 msgr2=0x55b3294da2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:50.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f57593fd740 1 -- 172.21.15.17:0/10357652 shutdown_connections 2024-03-24T03:00:50.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f57593fd740 1 -- 172.21.15.17:0/10357652 wait complete. 2024-03-24T03:00:50.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.128+0000 7f57593fd740 20 client.15519 trim_cache size 0 max 16384 2024-03-24T03:00:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.145+0000 7f57593fd740 1 Processor -- start 2024-03-24T03:00:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.145+0000 7f57593fd740 1 -- start start 2024-03-24T03:00:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.145+0000 7f57593fd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294e8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.145+0000 7f57593fd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.145+0000 7f57593fd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.145+0000 7f57593fd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3294f5cb0 con 0x55b3294df740 2024-03-24T03:00:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.145+0000 7f57593fd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3294f6310 con 0x55b3294e52a0 2024-03-24T03:00:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.145+0000 7f57593fd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3294f6970 con 0x55b3294e0130 2024-03-24T03:00:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f5754913640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f5754913640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294e8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42102/0 (socket says 172.21.15.17:42102) 2024-03-24T03:00:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f5754913640 1 -- 172.21.15.17:0/2622160664 learned_addr learned my addr 172.21.15.17:0/2622160664 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f5755114640 1 --2- 172.21.15.17:0/2622160664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f574ffff640 1 --2- 172.21.15.17:0/2622160664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f5754913640 1 -- 172.21.15.17:0/2622160664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 msgr2=0x55b3294e8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f5754913640 1 --2- 172.21.15.17:0/2622160664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294e8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f5754913640 1 -- 172.21.15.17:0/2622160664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 msgr2=0x55b3294f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f5754913640 1 --2- 172.21.15.17:0/2622160664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f5754913640 1 -- 172.21.15.17:0/2622160664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3292236b0 con 0x55b3294df740 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f574ffff640 1 --2- 172.21.15.17:0/2622160664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294e8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f5754913640 1 --2- 172.21.15.17:0/2622160664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294e8770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f57480bd190 tx=0x7f574802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddade2a08f3f596b server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f57367fc640 1 -- 172.21.15.17:0/2622160664 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57480bcd40 con 0x55b3294df740 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f57367fc640 1 -- 172.21.15.17:0/2622160664 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5748034920 con 0x55b3294df740 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.146+0000 7f57367fc640 1 -- 172.21.15.17:0/2622160664 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57480bd250 con 0x55b3294df740 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 -- 172.21.15.17:0/2622160664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 msgr2=0x55b3294e8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 --2- 172.21.15.17:0/2622160664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294e8770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f57480bd190 tx=0x7f574802c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 -- 172.21.15.17:0/2622160664 shutdown_connections 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 --2- 172.21.15.17:0/2622160664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 --2- 172.21.15.17:0/2622160664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294e8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 --2- 172.21.15.17:0/2622160664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294e8770 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 -- 172.21.15.17:0/2622160664 >> 172.21.15.17:0/2622160664 conn(0x55b3294a7a60 msgr2=0x55b3294dabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 -- 172.21.15.17:0/2622160664 shutdown_connections 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 -- 172.21.15.17:0/2622160664 wait complete. 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 -- start start 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 Processor -- start 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3294f7630 con 0x55b3294df740 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3294f6fd0 con 0x55b3294e52a0 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3294f7c90 con 0x55b3294e0130 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.147+0000 7f57593fd740 10 client.0 _ll_register_callbacks cb 0x7f5748040f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-24T03:00:50.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f5754913640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f5754913640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294ecfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42110/0 (socket says 172.21.15.17:42110) 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f5754913640 1 -- 172.21.15.17:0/1793643485 learned_addr learned my addr 172.21.15.17:0/1793643485 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f5755114640 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f574ffff640 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f5754913640 1 -- 172.21.15.17:0/1793643485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 msgr2=0x55b3294ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f5754913640 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f5754913640 1 -- 172.21.15.17:0/1793643485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 msgr2=0x55b3294f0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f5754913640 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f5754913640 1 -- 172.21.15.17:0/1793643485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3294f0b10 con 0x55b3294df740 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f5754913640 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294ecfc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f57480324c0 tx=0x7f57480b9870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b48564e8714d47b server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f572ffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f572ffff640 1 -- 172.21.15.17:0/1793643485 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57480bcd40 con 0x55b3294df740 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f572ffff640 1 -- 172.21.15.17:0/1793643485 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f5748032890 con 0x55b3294df740 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f57593fd740 1 -- 172.21.15.17:0/1793643485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5748034920 con 0x55b3294df740 2024-03-24T03:00:50.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f572ffff640 1 -- 172.21.15.17:0/1793643485 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57480c5910 con 0x55b3294df740 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f57593fd740 10 client.15525 fetch_fsmap learned FSMap version 28 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f57593fd740 1 -- 172.21.15.17:0/1793643485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b3294f0b10 con 0x55b3294df740 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.148+0000 7f572ffff640 1 -- 172.21.15.17:0/1793643485 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5748034920 con 0x55b3294df740 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f572ffff640 1 -- 172.21.15.17:0/1793643485 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f574803c040 con 0x55b3294df740 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 10 client.15525 fetch_fsmap finished waiting for FSMap version 28 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 10 client.15525 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 20 client.15525 populate_metadata read hostname 'smithi017' 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55b329555c80 0x55b329551c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 4 client.15525 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 1 -- 172.21.15.17:0/1793643485 --> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b32944c4b0 con 0x55b329555c80 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f574ffff640 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55b329555c80 0x55b329551c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f572ffff640 10 client.15525 ms_handle_connect on v2:172.21.15.188:6838/81409664 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f574ffff640 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55b329555c80 0x55b329551c70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dbb921f540024d2 server_cookie=f1716eeed69227e2 in_seq=0 out_seq=0 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f572ffff640 1 -- 172.21.15.17:0/1793643485 <== mds.0 v2:172.21.15.188:6838/81409664 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b32944c4b0 con 0x55b329555c80 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f572ffff640 10 client.15525 handle_command_reply: tid=0 2024-03-24T03:00:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 1 client.15525 shutdown 2024-03-24T03:00:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 1 -- 172.21.15.17:0/1793643485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 msgr2=0x55b3294ecfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294ecfc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f57480324c0 tx=0x7f57480b9870 comp rx=0 tx=0).stop 2024-03-24T03:00:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 1 -- 172.21.15.17:0/1793643485 shutdown_connections 2024-03-24T03:00:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3294e52a0 0x55b3294f0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3294e0130 0x55b3294ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.188:6838/81409664,v1:172.21.15.188:6839/81409664] conn(0x55b329555c80 0x55b329551c70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.149+0000 7f57593fd740 1 --2- 172.21.15.17:0/1793643485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b3294df740 0x55b3294ecfc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f572ffff640 0 client.15525 ms_handle_reset on v2:172.21.15.188:6838/81409664 2024-03-24T03:00:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f57593fd740 1 -- 172.21.15.17:0/1793643485 >> 172.21.15.17:0/1793643485 conn(0x55b3294a7a60 msgr2=0x55b3294e1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f57593fd740 1 -- 172.21.15.17:0/1793643485 shutdown_connections 2024-03-24T03:00:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f57593fd740 1 -- 172.21.15.17:0/1793643485 wait complete. 2024-03-24T03:00:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f57593fd740 20 client.15525 trim_cache size 0 max 16384 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "2a682800-e989-11ee-95cd-87774f69a715", 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24529, 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 132.77760283699999, 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 28, 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 93, 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 147.75105204100001 2024-03-24T03:00:50.157 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:00:50.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f5756b77640 1 -- 172.21.15.17:0/620359258 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f57380a2800 msgr2=0x7f57380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f5756b77640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f57380a2800 0x7f57380a4cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5740027000 tx=0x7f5740024000 comp rx=0 tx=0).stop 2024-03-24T03:00:50.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f5756b77640 1 -- 172.21.15.17:0/620359258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57501005b0 msgr2=0x7f575006a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:50.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f5756b77640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57501005b0 0x7f575006a7e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f574802c630 tx=0x7f5748032f20 comp rx=0 tx=0).stop 2024-03-24T03:00:50.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f5756b77640 1 -- 172.21.15.17:0/620359258 shutdown_connections 2024-03-24T03:00:50.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f5756b77640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f57380a2800 0x7f57380a4cc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f5756b77640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5750102160 0x7f575019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f5756b77640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57501017a0 0x7f575019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f5756b77640 1 --2- 172.21.15.17:0/620359258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57501005b0 0x7f575006a7e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:50.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f5756b77640 1 -- 172.21.15.17:0/620359258 >> 172.21.15.17:0/620359258 conn(0x7f57500fbb40 msgr2=0x7f5750104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:50.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f5756b77640 1 -- 172.21.15.17:0/620359258 shutdown_connections 2024-03-24T03:00:50.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:50.150+0000 7f5756b77640 1 -- 172.21.15.17:0/620359258 wait complete. 2024-03-24T03:00:50.161 INFO:teuthology.run_tasks:Running task kclient... 2024-03-24T03:00:50.178 INFO:tasks.kclient:Mounting kernel clients... 2024-03-24T03:00:50.178 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-24T03:00:50.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:50.179 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-03-24T03:00:50.193 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-24T03:00:50.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:50.194 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-24T03:00:50.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:50.261 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link delete ceph-brx 2024-03-24T03:00:50.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:49 smithi188 ceph-mon[36206]: pgmap v362: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 55 B/s, 0 objects/s recovering 2024-03-24T03:00:50.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:49 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3270391664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:50.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:49 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1865167529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:49 smithi019 ceph-mon[35944]: pgmap v362: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 55 B/s, 0 objects/s recovering 2024-03-24T03:00:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:49 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3270391664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:00:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:49 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1865167529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:00:50.360 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-03-24T03:00:50.361 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-24T03:00:50.361 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-24T03:00:50.361 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-24T03:00:50.606 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:00:52.111 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:51 smithi017 ceph-mon[31926]: pgmap v363: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-24T03:00:52.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:51 smithi188 ceph-mon[36206]: pgmap v363: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-24T03:00:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:51 smithi019 ceph-mon[35944]: pgmap v363: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-24T03:00:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.365+0000 7f70e8ab1700 1 -- 172.21.15.17:0/1690133998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e4101cd0 msgr2=0x7f70e4105cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.365+0000 7f70e8ab1700 1 --2- 172.21.15.17:0/1690133998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e4101cd0 0x7f70e4105cd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009990 tx=0x7f70d8009ca0 comp rx=0 tx=0).stop 2024-03-24T03:00:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.365+0000 7f70e8ab1700 1 -- 172.21.15.17:0/1690133998 shutdown_connections 2024-03-24T03:00:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.365+0000 7f70e8ab1700 1 --2- 172.21.15.17:0/1690133998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70e41063b0 0x7f70e410df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.365+0000 7f70e8ab1700 1 --2- 172.21.15.17:0/1690133998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e4101320 0x7f70e4101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.365+0000 7f70e8ab1700 1 --2- 172.21.15.17:0/1690133998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e4101cd0 0x7f70e4105cd0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.365+0000 7f70e8ab1700 1 -- 172.21.15.17:0/1690133998 >> 172.21.15.17:0/1690133998 conn(0x7f70e40770b0 msgr2=0x7f70e40774c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:52.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.365+0000 7f70e8ab1700 1 -- 172.21.15.17:0/1690133998 shutdown_connections 2024-03-24T03:00:52.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.365+0000 7f70e8ab1700 1 -- 172.21.15.17:0/1690133998 wait complete. 2024-03-24T03:00:52.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e8ab1700 1 Processor -- start 2024-03-24T03:00:52.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e8ab1700 1 -- start start 2024-03-24T03:00:52.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e8ab1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e4101320 0x7f70e419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:52.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e8ab1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e4101cd0 0x7f70e419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:52.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e8ab1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70e41063b0 0x7f70e41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:52.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e8ab1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4112490 con 0x7f70e4101320 2024-03-24T03:00:52.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e8ab1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4112310 con 0x7f70e4101cd0 2024-03-24T03:00:52.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e8ab1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4112610 con 0x7f70e41063b0 2024-03-24T03:00:52.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e4101cd0 0x7f70e419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:52.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e4101320 0x7f70e419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:52.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e4101cd0 0x7f70e419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:40808/0 (socket says 172.21.15.17:40808) 2024-03-24T03:00:52.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e1d9b700 1 -- 172.21.15.17:0/3528685786 learned_addr learned my addr 172.21.15.17:0/3528685786 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:52.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e259c700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e4101320 0x7f70e419e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54988/0 (socket says 172.21.15.17:54988) 2024-03-24T03:00:52.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.366+0000 7f70e2d9d700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70e41063b0 0x7f70e41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:52.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.367+0000 7f70e1d9b700 1 -- 172.21.15.17:0/3528685786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70e41063b0 msgr2=0x7f70e41987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:52.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.367+0000 7f70e1d9b700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70e41063b0 0x7f70e41987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:52.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.367+0000 7f70e1d9b700 1 -- 172.21.15.17:0/3528685786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e4101320 msgr2=0x7f70e419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:52.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.367+0000 7f70e1d9b700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e4101320 0x7f70e419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:52.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.367+0000 7f70e1d9b700 1 -- 172.21.15.17:0/3528685786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e4198f90 con 0x7f70e4101cd0 2024-03-24T03:00:52.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.367+0000 7f70e259c700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e4101320 0x7f70e419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:52.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.367+0000 7f70e1d9b700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e4101cd0 0x7f70e419eb70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f70d8000c00 tx=0x7f70d800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:52.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.367+0000 7f70d37fe700 1 -- 172.21.15.17:0/3528685786 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8027070 con 0x7f70e4101cd0 2024-03-24T03:00:52.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.367+0000 7f70d37fe700 1 -- 172.21.15.17:0/3528685786 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70d8003d70 con 0x7f70e4101cd0 2024-03-24T03:00:52.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.367+0000 7f70e8ab1700 1 -- 172.21.15.17:0/3528685786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e4199220 con 0x7f70e4101cd0 2024-03-24T03:00:52.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.367+0000 7f70d37fe700 1 -- 172.21.15.17:0/3528685786 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d800f460 con 0x7f70e4101cd0 2024-03-24T03:00:52.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.367+0000 7f70e8ab1700 1 -- 172.21.15.17:0/3528685786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e410e670 con 0x7f70e4101cd0 2024-03-24T03:00:52.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.368+0000 7f70e8ab1700 1 -- 172.21.15.17:0/3528685786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e41027b0 con 0x7f70e4101cd0 2024-03-24T03:00:52.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.369+0000 7f70d37fe700 1 -- 172.21.15.17:0/3528685786 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f70d802a070 con 0x7f70e4101cd0 2024-03-24T03:00:52.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.370+0000 7f70d37fe700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f70cc0a2890 0x7f70cc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:52.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.370+0000 7f70e259c700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f70cc0a2890 0x7f70cc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:52.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.370+0000 7f70d37fe700 1 -- 172.21.15.17:0/3528685786 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f70d80cfcd0 con 0x7f70e4101cd0 2024-03-24T03:00:52.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.370+0000 7f70e259c700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f70cc0a2890 0x7f70cc0a4d50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009730 tx=0x7f70d4012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:52.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.374+0000 7f70d37fe700 1 -- 172.21.15.17:0/3528685786 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f70d8013070 con 0x7f70e4101cd0 2024-03-24T03:00:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.627+0000 7f70e8ab1700 1 -- 172.21.15.17:0/3528685786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f70e4068e50 con 0x7f70e4101cd0 2024-03-24T03:00:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.627+0000 7f70d37fe700 1 -- 172.21.15.17:0/3528685786 <== mon.2 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7f70d809c960 con 0x7f70e4101cd0 2024-03-24T03:00:52.628 INFO:teuthology.orchestra.run.smithi017.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-24T03:00:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.629+0000 7f70e8ab1700 1 -- 172.21.15.17:0/3528685786 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f70cc0a2890 msgr2=0x7f70cc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.629+0000 7f70e8ab1700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f70cc0a2890 0x7f70cc0a4d50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009730 tx=0x7f70d4012040 comp rx=0 tx=0).stop 2024-03-24T03:00:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.629+0000 7f70e8ab1700 1 -- 172.21.15.17:0/3528685786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e4101cd0 msgr2=0x7f70e419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.629+0000 7f70e8ab1700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e4101cd0 0x7f70e419eb70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f70d8000c00 tx=0x7f70d800bab0 comp rx=0 tx=0).stop 2024-03-24T03:00:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.629+0000 7f70e8ab1700 1 -- 172.21.15.17:0/3528685786 shutdown_connections 2024-03-24T03:00:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.629+0000 7f70e8ab1700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f70cc0a2890 0x7f70cc0a4d50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.629+0000 7f70e8ab1700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70e41063b0 0x7f70e41987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.629+0000 7f70e8ab1700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e4101320 0x7f70e419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.629+0000 7f70e8ab1700 1 --2- 172.21.15.17:0/3528685786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e4101cd0 0x7f70e419eb70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.629+0000 7f70e8ab1700 1 -- 172.21.15.17:0/3528685786 >> 172.21.15.17:0/3528685786 conn(0x7f70e40770b0 msgr2=0x7f70e4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.629+0000 7f70e8ab1700 1 -- 172.21.15.17:0/3528685786 shutdown_connections 2024-03-24T03:00:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:52.630+0000 7f70e8ab1700 1 -- 172.21.15.17:0/3528685786 wait complete. 2024-03-24T03:00:53.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:52 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3528685786' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T03:00:53.294 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-24T03:00:53.294 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-24T03:00:53.294 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi017.front.sepia.ceph.com 2024-03-24T03:00:53.294 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-24T03:00:53.294 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-24T03:00:53.295 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-24T03:00:53.295 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-24T03:00:53.295 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-24T03:00:53.295 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-24T03:00:53.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:53.295 DEBUG:teuthology.orchestra.run.smithi017:> ip addr 2024-03-24T03:00:53.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:52 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3528685786' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T03:00:53.311 INFO:teuthology.orchestra.run.smithi017.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout: inet 127.0.0.1/8 scope host lo 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 ::1/128 scope host 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:00 brd ff:ff:ff:ff:ff:ff 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f0 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:01 brd ff:ff:ff:ff:ff:ff 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f1 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-24T03:00:53.312 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:92 brd ff:ff:ff:ff:ff:ff 2024-03-24T03:00:53.313 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f0 2024-03-24T03:00:53.313 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.21.15.17/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-24T03:00:53.313 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft 42064sec preferred_lft 42064sec 2024-03-24T03:00:53.313 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::ec4:7aff:fe59:d992/64 scope link noprefixroute 2024-03-24T03:00:53.313 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-24T03:00:53.313 INFO:teuthology.orchestra.run.smithi017.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T03:00:53.313 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:93 brd ff:ff:ff:ff:ff:ff 2024-03-24T03:00:53.313 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f1 2024-03-24T03:00:53.313 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-24T03:00:53.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:53.313 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-24T03:00:53.313 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-24T03:00:53.314 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add name ceph-brx type bridge 2024-03-24T03:00:53.314 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr flush dev ceph-brx 2024-03-24T03:00:53.314 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set ceph-brx up 2024-03-24T03:00:53.314 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-24T03:00:53.314 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-24T03:00:53.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:52 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3528685786' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T03:00:53.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:53.425 DEBUG:teuthology.orchestra.run.smithi017:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-24T03:00:53.450 INFO:teuthology.orchestra.run.smithi017.stdout:1 2024-03-24T03:00:53.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:53.452 DEBUG:teuthology.orchestra.run.smithi017:> route 2024-03-24T03:00:53.508 INFO:teuthology.orchestra.run.smithi017.stdout:Kernel IP routing table 2024-03-24T03:00:53.508 INFO:teuthology.orchestra.run.smithi017.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-24T03:00:53.508 INFO:teuthology.orchestra.run.smithi017.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-24T03:00:53.508 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-24T03:00:53.508 INFO:teuthology.orchestra.run.smithi017.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-24T03:00:53.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:53.509 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-24T03:00:53.509 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-24T03:00:53.509 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-24T03:00:53.509 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-24T03:00:53.509 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-24T03:00:53.509 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-24T03:00:53.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:53.604 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-03-24T03:00:53.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:53.619 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list-id 2024-03-24T03:00:53.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:53.673 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-24T03:00:53.673 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-24T03:00:53.673 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-24T03:00:53.673 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-24T03:00:53.673 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-24T03:00:53.755 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-24T03:00:53.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:53.756 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-24T03:00:53.756 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-24T03:00:53.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-24T03:00:53.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-24T03:00:53.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-24T03:00:53.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-24T03:00:53.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-24T03:00:53.756 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-24T03:00:53.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:53.882 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-24T03:00:53.882 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-24T03:00:53.882 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set brx.0 up 2024-03-24T03:00:53.882 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set dev brx.0 master ceph-brx 2024-03-24T03:00:53.882 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-24T03:00:53.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-24T03:00:53.925 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-24T03:00:53.979 INFO:teuthology.orchestra.run.smithi017.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-24T03:00:53.980 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-24T03:00:53.980 DEBUG:teuthology.orchestra.run.smithi017:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-24T03:00:54.034 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-24T03:00:54.034 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-24T03:00:54.034 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/04bb16c3-438e-4302-875a-85564d3f557c 2024-03-24T03:00:54.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:54.035 DEBUG:teuthology.orchestra.run.smithi017:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/04bb16c3-438e-4302-875a-85564d3f557c /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-03-24T03:00:54.097 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:53 smithi017 ceph-mon[31926]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-24T03:00:54.117 INFO:teuthology.orchestra.run.smithi017.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-03-24T03:00:54.117 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-03-24T03:00:54.248 INFO:teuthology.orchestra.run.smithi017.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-03-24T03:00:54.248 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-03-24T03:00:54.249 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: resolved to: "172.21.15.17:6789,172.21.15.19:6789,172.21.15.188:6789" 2024-03-24T03:00:54.249 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: trying mount with new device syntax: 0@2a682800-e989-11ee-95cd-87774f69a715.cephfs=/volumes/qa/sv_1/04bb16c3-438e-4302-875a-85564d3f557c 2024-03-24T03:00:54.249 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.17:6789/172.21.15.19:6789/172.21.15.188:6789" will pass to kernel 2024-03-24T03:00:54.250 INFO:teuthology.orchestra.run.smithi017.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-24T03:00:54.250 INFO:teuthology.orchestra.run.smithi017.stdout: You just mounted a file system that supports labels which does not 2024-03-24T03:00:54.250 INFO:teuthology.orchestra.run.smithi017.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-24T03:00:54.250 INFO:teuthology.orchestra.run.smithi017.stdout: applications will generate AVC messages and not be allowed access to 2024-03-24T03:00:54.250 INFO:teuthology.orchestra.run.smithi017.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-24T03:00:54.251 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-24T03:00:54.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:54.252 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-24T03:00:54.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:53 smithi188 ceph-mon[36206]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-24T03:00:54.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:54.320 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T03:00:54.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:53 smithi019 ceph-mon[35944]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-24T03:00:54.376 INFO:teuthology.orchestra.run.smithi017.stdout:client15534 2024-03-24T03:00:54.376 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T03:00:54.376 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T03:00:54.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:54.432 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T03:00:54.487 INFO:teuthology.orchestra.run.smithi017.stdout:client15534 2024-03-24T03:00:54.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:54.488 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T03:00:54.543 INFO:teuthology.orchestra.run.smithi017.stdout:client15534 2024-03-24T03:00:54.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:00:54.544 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/2a682800-e989-11ee-95cd-87774f69a715.client15534/status') 2024-03-24T03:00:54.610 INFO:teuthology.orchestra.run.smithi017.stdout:instance: client.15534 (3)192.168.0.1:0/1519475664 2024-03-24T03:00:54.610 INFO:teuthology.orchestra.run.smithi017.stdout:blocklisted: false 2024-03-24T03:00:54.610 INFO:teuthology.orchestra.run.smithi017.stderr:0+1 records in 2024-03-24T03:00:54.610 INFO:teuthology.orchestra.run.smithi017.stderr:0+1 records out 2024-03-24T03:00:54.610 INFO:teuthology.orchestra.run.smithi017.stderr:70 bytes copied, 2.3605e-05 s, 3.0 MB/s 2024-03-24T03:00:54.611 INFO:teuthology.run_tasks:Running task exec... 2024-03-24T03:00:54.621 INFO:teuthology.task.exec:Executing custom commands... 2024-03-24T03:00:54.622 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2024-03-24T03:00:54.622 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-24T03:00:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.755+0000 7f3cadc44640 1 -- 172.21.15.17:0/2924715214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca8104a70 msgr2=0x7f3ca8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:54.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.755+0000 7f3cadc44640 1 --2- 172.21.15.17:0/2924715214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca8104a70 0x7f3ca8106e60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009900 tx=0x7f3c9c02bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.755+0000 7f3cadc44640 1 -- 172.21.15.17:0/2924715214 shutdown_connections 2024-03-24T03:00:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.755+0000 7f3cadc44640 1 --2- 172.21.15.17:0/2924715214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca81073a0 0x7f3ca81097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.755+0000 7f3cadc44640 1 --2- 172.21.15.17:0/2924715214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca8104a70 0x7f3ca8106e60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.755+0000 7f3cadc44640 1 --2- 172.21.15.17:0/2924715214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca8102140 0x7f3ca8104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.755+0000 7f3cadc44640 1 -- 172.21.15.17:0/2924715214 >> 172.21.15.17:0/2924715214 conn(0x7f3ca80fbb30 msgr2=0x7f3ca80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.755+0000 7f3cadc44640 1 -- 172.21.15.17:0/2924715214 shutdown_connections 2024-03-24T03:00:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.755+0000 7f3cadc44640 1 -- 172.21.15.17:0/2924715214 wait complete. 2024-03-24T03:00:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3cadc44640 1 Processor -- start 2024-03-24T03:00:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3cadc44640 1 -- start start 2024-03-24T03:00:54.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3cadc44640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca8102140 0x7f3ca8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3cadc44640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca8104a70 0x7f3ca8197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3cadc44640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca81073a0 0x7f3ca819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:54.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3cadc44640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca810d330 con 0x7f3ca81073a0 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3cadc44640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca810d1b0 con 0x7f3ca8102140 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3cadc44640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca810d4b0 con 0x7f3ca8104a70 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca81073a0 0x7f3ca819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3ca77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca8102140 0x7f3ca8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca81073a0 0x7f3ca819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55012/0 (socket says 172.21.15.17:55012) 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3ca7fff640 1 -- 172.21.15.17:0/2773533141 learned_addr learned my addr 172.21.15.17:0/2773533141 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3ca6ffd640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca8104a70 0x7f3ca8197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3ca7fff640 1 -- 172.21.15.17:0/2773533141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca8104a70 msgr2=0x7f3ca8197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3ca7fff640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca8104a70 0x7f3ca8197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3ca7fff640 1 -- 172.21.15.17:0/2773533141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca8102140 msgr2=0x7f3ca8197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3ca7fff640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca8102140 0x7f3ca8197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3ca7fff640 1 -- 172.21.15.17:0/2773533141 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca819f570 con 0x7f3ca81073a0 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.756+0000 7f3ca77fe640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca8102140 0x7f3ca8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:00:54.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.757+0000 7f3ca7fff640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca81073a0 0x7f3ca819ef00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000efd0 tx=0x7f3ca000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.757+0000 7f3ca4ff9640 1 -- 172.21.15.17:0/2773533141 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0003b70 con 0x7f3ca81073a0 2024-03-24T03:00:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.757+0000 7f3ca4ff9640 1 -- 172.21.15.17:0/2773533141 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ca0003d10 con 0x7f3ca81073a0 2024-03-24T03:00:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.757+0000 7f3cadc44640 1 -- 172.21.15.17:0/2773533141 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca819f860 con 0x7f3ca81073a0 2024-03-24T03:00:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.757+0000 7f3ca4ff9640 1 -- 172.21.15.17:0/2773533141 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00109f0 con 0x7f3ca81073a0 2024-03-24T03:00:54.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.757+0000 7f3cadc44640 1 -- 172.21.15.17:0/2773533141 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca819fd30 con 0x7f3ca81073a0 2024-03-24T03:00:54.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.758+0000 7f3cadc44640 1 -- 172.21.15.17:0/2773533141 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca80fd620 con 0x7f3ca81073a0 2024-03-24T03:00:54.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.759+0000 7f3ca4ff9640 1 -- 172.21.15.17:0/2773533141 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3ca0004020 con 0x7f3ca81073a0 2024-03-24T03:00:54.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.760+0000 7f3ca4ff9640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3c900a28a0 0x7f3c900a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:54.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.760+0000 7f3ca77fe640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3c900a28a0 0x7f3c900a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:54.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.760+0000 7f3ca4ff9640 1 -- 172.21.15.17:0/2773533141 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f3ca00cf310 con 0x7f3ca81073a0 2024-03-24T03:00:54.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.760+0000 7f3ca77fe640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3c900a28a0 0x7f3c900a4d60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3c980095f0 tx=0x7f3c98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:54.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.764+0000 7f3ca4ff9640 1 -- 172.21.15.17:0/2773533141 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ca009bf70 con 0x7f3ca81073a0 2024-03-24T03:00:54.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:54.988+0000 7f3cadc44640 1 -- 172.21.15.17:0/2773533141 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f3ca80622a0 con 0x7f3ca81073a0 2024-03-24T03:00:55.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.926+0000 7f3ca4ff9640 1 -- 172.21.15.17:0/2773533141 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3ca009b870 con 0x7f3ca81073a0 2024-03-24T03:00:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.929+0000 7f3cadc44640 1 -- 172.21.15.17:0/2773533141 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3c900a28a0 msgr2=0x7f3c900a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.929+0000 7f3cadc44640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3c900a28a0 0x7f3c900a4d60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3c980095f0 tx=0x7f3c98005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.929+0000 7f3cadc44640 1 -- 172.21.15.17:0/2773533141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca81073a0 msgr2=0x7f3ca819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.929+0000 7f3cadc44640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca81073a0 0x7f3ca819ef00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000efd0 tx=0x7f3ca000c6a0 comp rx=0 tx=0).stop 2024-03-24T03:00:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.930+0000 7f3cadc44640 1 -- 172.21.15.17:0/2773533141 shutdown_connections 2024-03-24T03:00:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.930+0000 7f3cadc44640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f3c900a28a0 0x7f3c900a4d60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.930+0000 7f3cadc44640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca81073a0 0x7f3ca819ef00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.930+0000 7f3cadc44640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca8104a70 0x7f3ca8197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:55.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.930+0000 7f3cadc44640 1 --2- 172.21.15.17:0/2773533141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca8102140 0x7f3ca8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:55.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.930+0000 7f3cadc44640 1 -- 172.21.15.17:0/2773533141 >> 172.21.15.17:0/2773533141 conn(0x7f3ca80fbb30 msgr2=0x7f3ca8102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:55.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.930+0000 7f3cadc44640 1 -- 172.21.15.17:0/2773533141 shutdown_connections 2024-03-24T03:00:55.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:55.930+0000 7f3cadc44640 1 -- 172.21.15.17:0/2773533141 wait complete. 2024-03-24T03:00:55.943 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-24T03:00:56.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.056+0000 7f53b25e1640 1 -- 172.21.15.17:0/3167286505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ac1073a0 msgr2=0x7f53ac1097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:56.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.056+0000 7f53b25e1640 1 --2- 172.21.15.17:0/3167286505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ac1073a0 0x7f53ac1097f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009ca0 tx=0x7f53a402c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.056+0000 7f53b25e1640 1 -- 172.21.15.17:0/3167286505 shutdown_connections 2024-03-24T03:00:56.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.056+0000 7f53b25e1640 1 --2- 172.21.15.17:0/3167286505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ac1073a0 0x7f53ac1097f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.056+0000 7f53b25e1640 1 --2- 172.21.15.17:0/3167286505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ac104a70 0x7f53ac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.056+0000 7f53b25e1640 1 --2- 172.21.15.17:0/3167286505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53ac102140 0x7f53ac104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.056+0000 7f53b25e1640 1 -- 172.21.15.17:0/3167286505 >> 172.21.15.17:0/3167286505 conn(0x7f53ac0fbb30 msgr2=0x7f53ac0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.056+0000 7f53b25e1640 1 -- 172.21.15.17:0/3167286505 shutdown_connections 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.056+0000 7f53b25e1640 1 -- 172.21.15.17:0/3167286505 wait complete. 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.056+0000 7f53b25e1640 1 Processor -- start 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.056+0000 7f53b25e1640 1 -- start start 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.056+0000 7f53b25e1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ac102140 0x7f53ac197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53b25e1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53ac104a70 0x7f53ac197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53b25e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ac1073a0 0x7f53ac19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53b25e1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac10d360 con 0x7f53ac102140 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53b25e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac10d1e0 con 0x7f53ac1073a0 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53b25e1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac10d4e0 con 0x7f53ac104a70 2024-03-24T03:00:56.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53abfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ac102140 0x7f53ac197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:56.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53abfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ac102140 0x7f53ac197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55042/0 (socket says 172.21.15.17:55042) 2024-03-24T03:00:56.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53abfff640 1 -- 172.21.15.17:0/1777450218 learned_addr learned my addr 172.21.15.17:0/1777450218 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:56.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53b0b7e640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ac1073a0 0x7f53ac19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:56.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53ab7fe640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53ac104a70 0x7f53ac197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:56.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53abfff640 1 -- 172.21.15.17:0/1777450218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53ac104a70 msgr2=0x7f53ac197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:56.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53abfff640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53ac104a70 0x7f53ac197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53abfff640 1 -- 172.21.15.17:0/1777450218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ac1073a0 msgr2=0x7f53ac19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:56.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53abfff640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ac1073a0 0x7f53ac19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53abfff640 1 -- 172.21.15.17:0/1777450218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac19f560 con 0x7f53ac102140 2024-03-24T03:00:56.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53abfff640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ac102140 0x7f53ac197940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f539c00e9f0 tx=0x7f539c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:56.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53a97fa640 1 -- 172.21.15.17:0/1777450218 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c00cd80 con 0x7f53ac102140 2024-03-24T03:00:56.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53b25e1640 1 -- 172.21.15.17:0/1777450218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ac19f850 con 0x7f53ac102140 2024-03-24T03:00:56.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53a97fa640 1 -- 172.21.15.17:0/1777450218 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f539c005500 con 0x7f53ac102140 2024-03-24T03:00:56.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53a97fa640 1 -- 172.21.15.17:0/1777450218 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c010690 con 0x7f53ac102140 2024-03-24T03:00:56.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.057+0000 7f53b25e1640 1 -- 172.21.15.17:0/1777450218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ac19fd50 con 0x7f53ac102140 2024-03-24T03:00:56.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.059+0000 7f53b25e1640 1 -- 172.21.15.17:0/1777450218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ac0fd620 con 0x7f53ac102140 2024-03-24T03:00:56.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.060+0000 7f53a97fa640 1 -- 172.21.15.17:0/1777450218 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f539c0046c0 con 0x7f53ac102140 2024-03-24T03:00:56.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.060+0000 7f53a97fa640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f53940a28a0 0x7f53940a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:56.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.060+0000 7f53ab7fe640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f53940a28a0 0x7f53940a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:56.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.060+0000 7f53a97fa640 1 -- 172.21.15.17:0/1777450218 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f539c0cf290 con 0x7f53ac102140 2024-03-24T03:00:56.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.061+0000 7f53ab7fe640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f53940a28a0 0x7f53940a4d60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f53ac198e50 tx=0x7f53a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:56.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.065+0000 7f53a97fa640 1 -- 172.21.15.17:0/1777450218 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f539c09bef0 con 0x7f53ac102140 2024-03-24T03:00:56.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:55 smithi017 ceph-mon[31926]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T03:00:56.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:55 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2773533141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-24T03:00:56.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.277+0000 7f53b25e1640 1 -- 172.21.15.17:0/1777450218 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f53ac061b80 con 0x7f53940a28a0 2024-03-24T03:00:56.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.293+0000 7f53a97fa640 1 -- 172.21.15.17:0/1777450218 <== mgr.14152 v2:172.21.15.17:6800/2663792883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f53ac061b80 con 0x7f53940a28a0 2024-03-24T03:00:56.293 INFO:teuthology.orchestra.run.smithi017.stdout:{} 2024-03-24T03:00:56.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.295+0000 7f53b25e1640 1 -- 172.21.15.17:0/1777450218 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f53940a28a0 msgr2=0x7f53940a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:56.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.295+0000 7f53b25e1640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f53940a28a0 0x7f53940a4d60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f53ac198e50 tx=0x7f53a0005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:56.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.295+0000 7f53b25e1640 1 -- 172.21.15.17:0/1777450218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ac102140 msgr2=0x7f53ac197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:56.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.295+0000 7f53b25e1640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ac102140 0x7f53ac197940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f539c00e9f0 tx=0x7f539c00edf0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.295+0000 7f53b25e1640 1 -- 172.21.15.17:0/1777450218 shutdown_connections 2024-03-24T03:00:56.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.295+0000 7f53b25e1640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f53940a28a0 0x7f53940a4d60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.295+0000 7f53b25e1640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ac1073a0 0x7f53ac19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.295+0000 7f53b25e1640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53ac104a70 0x7f53ac197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.295+0000 7f53b25e1640 1 --2- 172.21.15.17:0/1777450218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ac102140 0x7f53ac197940 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.295+0000 7f53b25e1640 1 -- 172.21.15.17:0/1777450218 >> 172.21.15.17:0/1777450218 conn(0x7f53ac0fbb30 msgr2=0x7f53ac102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:56.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.295+0000 7f53b25e1640 1 -- 172.21.15.17:0/1777450218 shutdown_connections 2024-03-24T03:00:56.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.295+0000 7f53b25e1640 1 -- 172.21.15.17:0/1777450218 wait complete. 2024-03-24T03:00:56.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:55 smithi188 ceph-mon[36206]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T03:00:56.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:55 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2773533141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-24T03:00:56.306 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-24T03:00:56.317 INFO:teuthology.run_tasks:Running task exec... 2024-03-24T03:00:56.339 INFO:teuthology.task.exec:Executing custom commands... 2024-03-24T03:00:56.339 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2024-03-24T03:00:56.339 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-24T03:00:56.340 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:55 smithi019 ceph-mon[35944]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-24T03:00:56.340 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:55 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2773533141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-24T03:00:56.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.442+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/1614486137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f4102170 msgr2=0x7fe4f4108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.442+0000 7fe4fc3f4640 1 --2- 172.21.15.17:0/1614486137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f4102170 0x7fe4f4108a00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009ca0 tx=0x7fe4ec02c0f0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.442+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/1614486137 shutdown_connections 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.442+0000 7fe4fc3f4640 1 --2- 172.21.15.17:0/1614486137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f4102170 0x7fe4f4108a00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.442+0000 7fe4fc3f4640 1 --2- 172.21.15.17:0/1614486137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f41017b0 0x7fe4f4101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.442+0000 7fe4fc3f4640 1 --2- 172.21.15.17:0/1614486137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f41005c0 0x7fe4f41009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.442+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/1614486137 >> 172.21.15.17:0/1614486137 conn(0x7fe4f40fbb30 msgr2=0x7fe4f40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.442+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/1614486137 shutdown_connections 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.442+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/1614486137 wait complete. 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.442+0000 7fe4fc3f4640 1 Processor -- start 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4fc3f4640 1 -- start start 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4fc3f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f41005c0 0x7fe4f41978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4fc3f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f41017b0 0x7fe4f4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4fc3f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f4102170 0x7fe4f419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4fc3f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f410d360 con 0x7fe4f41017b0 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4fc3f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f410d1e0 con 0x7fe4f41005c0 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4fc3f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f410d4e0 con 0x7fe4f4102170 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4f998f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f41017b0 0x7fe4f4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4f998f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f41017b0 0x7fe4f4197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55068/0 (socket says 172.21.15.17:55068) 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4f998f640 1 -- 172.21.15.17:0/988880978 learned_addr learned my addr 172.21.15.17:0/988880978 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4fa190640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f41005c0 0x7fe4f41978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4fa991640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f4102170 0x7fe4f419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4f998f640 1 -- 172.21.15.17:0/988880978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f4102170 msgr2=0x7fe4f419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4f998f640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f4102170 0x7fe4f419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4f998f640 1 -- 172.21.15.17:0/988880978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f41005c0 msgr2=0x7fe4f41978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4f998f640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f41005c0 0x7fe4f41978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4f998f640 1 -- 172.21.15.17:0/988880978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f419f4e0 con 0x7fe4f41017b0 2024-03-24T03:00:56.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4f998f640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f41017b0 0x7fe4f4197e00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400e9f0 tx=0x7fe4e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:56.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4eb7fe640 1 -- 172.21.15.17:0/988880978 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e400cd80 con 0x7fe4f41017b0 2024-03-24T03:00:56.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4eb7fe640 1 -- 172.21.15.17:0/988880978 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4e4010910 con 0x7fe4f41017b0 2024-03-24T03:00:56.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4eb7fe640 1 -- 172.21.15.17:0/988880978 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4018990 con 0x7fe4f41017b0 2024-03-24T03:00:56.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.443+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/988880978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f419f7d0 con 0x7fe4f41017b0 2024-03-24T03:00:56.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.444+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/988880978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f419fd70 con 0x7fe4f41017b0 2024-03-24T03:00:56.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.444+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/988880978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f4107880 con 0x7fe4f41017b0 2024-03-24T03:00:56.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.446+0000 7fe4eb7fe640 1 -- 172.21.15.17:0/988880978 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe4e4010ac0 con 0x7fe4f41017b0 2024-03-24T03:00:56.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.446+0000 7fe4eb7fe640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe4e00a2920 0x7fe4e00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:56.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.446+0000 7fe4fa190640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe4e00a2920 0x7fe4e00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:56.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.446+0000 7fe4eb7fe640 1 -- 172.21.15.17:0/988880978 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fe4e40cf530 con 0x7fe4f41017b0 2024-03-24T03:00:56.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.447+0000 7fe4fa190640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe4e00a2920 0x7fe4e00a4de0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4101610 tx=0x7fe4f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:56.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.450+0000 7fe4eb7fe640 1 -- 172.21.15.17:0/988880978 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe4e409bb40 con 0x7fe4f41017b0 2024-03-24T03:00:56.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.685+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/988880978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe4f402d200 con 0x7fe4f41017b0 2024-03-24T03:00:56.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:56.281+0000 7f5ebb9d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:00:56.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:56.281+0000 7f5ebb9d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:00:56.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:56.281+0000 7f5ebb9d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:00:56.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:56.281+0000 7f5ebb9d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:00:56.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:56.281+0000 7f5ebb9d4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:00:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.932+0000 7fe4eb7fe640 1 -- 172.21.15.17:0/988880978 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe4e409fbe0 con 0x7fe4f41017b0 2024-03-24T03:00:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.935+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/988880978 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe4e00a2920 msgr2=0x7fe4e00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.935+0000 7fe4fc3f4640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe4e00a2920 0x7fe4e00a4de0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4101610 tx=0x7fe4f0005f90 comp rx=0 tx=0).stop 2024-03-24T03:00:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.935+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/988880978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f41017b0 msgr2=0x7fe4f4197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:56.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.935+0000 7fe4fc3f4640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f41017b0 0x7fe4f4197e00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400e9f0 tx=0x7fe4e400eeb0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.935+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/988880978 shutdown_connections 2024-03-24T03:00:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.935+0000 7fe4fc3f4640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fe4e00a2920 0x7fe4e00a4de0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.935+0000 7fe4fc3f4640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f4102170 0x7fe4f419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.935+0000 7fe4fc3f4640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f41017b0 0x7fe4f4197e00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.935+0000 7fe4fc3f4640 1 --2- 172.21.15.17:0/988880978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f41005c0 0x7fe4f41978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.935+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/988880978 >> 172.21.15.17:0/988880978 conn(0x7fe4f40fbb30 msgr2=0x7fe4f4104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.935+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/988880978 shutdown_connections 2024-03-24T03:00:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:56.935+0000 7fe4fc3f4640 1 -- 172.21.15.17:0/988880978 wait complete. 2024-03-24T03:00:56.952 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-24T03:00:57.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.062+0000 7f1396ff3640 1 -- 172.21.15.17:0/812685885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1390100910 msgr2=0x7f1390100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:57.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.062+0000 7f1396ff3640 1 --2- 172.21.15.17:0/812685885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1390100910 0x7f1390100d70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f137c009900 tx=0x7f137c02bcd0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.062+0000 7f1396ff3640 1 -- 172.21.15.17:0/812685885 shutdown_connections 2024-03-24T03:00:57.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.062+0000 7f1396ff3640 1 --2- 172.21.15.17:0/812685885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901012b0 0x7f1390101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.062+0000 7f1396ff3640 1 --2- 172.21.15.17:0/812685885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1390100910 0x7f1390100d70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.062+0000 7f1396ff3640 1 --2- 172.21.15.17:0/812685885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13900fffb0 0x7f13901003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.062+0000 7f1396ff3640 1 -- 172.21.15.17:0/812685885 >> 172.21.15.17:0/812685885 conn(0x7f13900fbb30 msgr2=0x7f13900fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:57.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.062+0000 7f1396ff3640 1 -- 172.21.15.17:0/812685885 shutdown_connections 2024-03-24T03:00:57.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.062+0000 7f1396ff3640 1 -- 172.21.15.17:0/812685885 wait complete. 2024-03-24T03:00:57.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.062+0000 7f1396ff3640 1 Processor -- start 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.062+0000 7f1396ff3640 1 -- start start 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1396ff3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13900fffb0 0x7f1390199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1396ff3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1390100910 0x7f139019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1396ff3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901012b0 0x7f13901a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1396ff3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139010f540 con 0x7f13900fffb0 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1396ff3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139010f3c0 con 0x7f13901012b0 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1396ff3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139010f6c0 con 0x7f1390100910 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1394d8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13900fffb0 0x7f1390199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1394d8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13900fffb0 0x7f1390199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55118/0 (socket says 172.21.15.17:55118) 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1394d8f640 1 -- 172.21.15.17:0/1371620206 learned_addr learned my addr 172.21.15.17:0/1371620206 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f138ffff640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1390100910 0x7f139019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1395590640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901012b0 0x7f13901a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1394d8f640 1 -- 172.21.15.17:0/1371620206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1390100910 msgr2=0x7f139019a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:57.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1394d8f640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1390100910 0x7f139019a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1394d8f640 1 -- 172.21.15.17:0/1371620206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901012b0 msgr2=0x7f13901a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:57.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1394d8f640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901012b0 0x7f13901a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1394d8f640 1 -- 172.21.15.17:0/1371620206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13901a1770 con 0x7f13900fffb0 2024-03-24T03:00:57.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1394d8f640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13900fffb0 0x7f1390199b50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f138400e9f0 tx=0x7f138400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:57.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f138dffb640 1 -- 172.21.15.17:0/1371620206 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138400cd80 con 0x7f13900fffb0 2024-03-24T03:00:57.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f1396ff3640 1 -- 172.21.15.17:0/1371620206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13901a1a60 con 0x7f13900fffb0 2024-03-24T03:00:57.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f138dffb640 1 -- 172.21.15.17:0/1371620206 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1384005500 con 0x7f13900fffb0 2024-03-24T03:00:57.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.063+0000 7f138dffb640 1 -- 172.21.15.17:0/1371620206 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384010690 con 0x7f13900fffb0 2024-03-24T03:00:57.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.064+0000 7f1396ff3640 1 -- 172.21.15.17:0/1371620206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13901a1f80 con 0x7f13900fffb0 2024-03-24T03:00:57.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.065+0000 7f1396ff3640 1 -- 172.21.15.17:0/1371620206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f139010ad00 con 0x7f13900fffb0 2024-03-24T03:00:57.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.065+0000 7f138dffb640 1 -- 172.21.15.17:0/1371620206 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 137307+0+0 (secure 0 0 0) 0x7f13840046c0 con 0x7f13900fffb0 2024-03-24T03:00:57.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.066+0000 7f138dffb640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f13800a28f0 0x7f13800a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:57.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.066+0000 7f138ffff640 1 -- 172.21.15.17:0/1371620206 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f13800a28f0 msgr2=0x7f13800a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2663792883 2024-03-24T03:00:57.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.066+0000 7f138dffb640 1 -- 172.21.15.17:0/1371620206 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7f13840cf2e0 con 0x7f13900fffb0 2024-03-24T03:00:57.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.066+0000 7f138ffff640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f13800a28f0 0x7f13800a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T03:00:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.070+0000 7f138dffb640 1 -- 172.21.15.17:0/1371620206 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f138409bf40 con 0x7f13900fffb0 2024-03-24T03:00:57.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:56 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2773533141' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-24T03:00:57.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:56 smithi017 ceph-mon[31926]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-24T03:00:57.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:56 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/988880978' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-24T03:00:57.206 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: ignoring --setuser ceph since I am not root 2024-03-24T03:00:57.206 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: ignoring --setgroup ceph since I am not root 2024-03-24T03:00:57.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.258+0000 7f1396ff3640 1 -- 172.21.15.17:0/1371620206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f139002d070 con 0x7f13900fffb0 2024-03-24T03:00:57.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.262+0000 7f138dffb640 1 -- 172.21.15.17:0/1371620206 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7f138409b840 con 0x7f13900fffb0 2024-03-24T03:00:57.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.264+0000 7f1396ff3640 1 -- 172.21.15.17:0/1371620206 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f13800a28f0 msgr2=0x7f13800a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T03:00:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.264+0000 7f1396ff3640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f13800a28f0 0x7f13800a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.264+0000 7f1396ff3640 1 -- 172.21.15.17:0/1371620206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13900fffb0 msgr2=0x7f1390199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.264+0000 7f1396ff3640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13900fffb0 0x7f1390199b50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f138400e9f0 tx=0x7f138400edf0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.264+0000 7f1396ff3640 1 -- 172.21.15.17:0/1371620206 shutdown_connections 2024-03-24T03:00:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.264+0000 7f1396ff3640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7f13800a28f0 0x7f13800a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.264+0000 7f1396ff3640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901012b0 0x7f13901a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.264+0000 7f1396ff3640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1390100910 0x7f139019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.264+0000 7f1396ff3640 1 --2- 172.21.15.17:0/1371620206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13900fffb0 0x7f1390199b50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.264+0000 7f1396ff3640 1 -- 172.21.15.17:0/1371620206 >> 172.21.15.17:0/1371620206 conn(0x7f13900fbb30 msgr2=0x7f13900fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.264+0000 7f1396ff3640 1 -- 172.21.15.17:0/1371620206 shutdown_connections 2024-03-24T03:00:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.264+0000 7f1396ff3640 1 -- 172.21.15.17:0/1371620206 wait complete. 2024-03-24T03:00:57.276 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-24T03:00:57.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:56 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2773533141' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-24T03:00:57.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:56 smithi188 ceph-mon[36206]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-24T03:00:57.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:56 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/988880978' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-24T03:00:57.298 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:56 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: ignoring --setuser ceph since I am not root 2024-03-24T03:00:57.298 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:56 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: ignoring --setgroup ceph since I am not root 2024-03-24T03:00:57.298 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:56 smithi188 ceph-mgr[38065]: -- 172.21.15.188:0/2512740031 <== mon.0 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555f79f4a5a0 con 0x555f79f1e800 2024-03-24T03:00:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:56 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2773533141' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-24T03:00:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:56 smithi019 ceph-mon[35944]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-24T03:00:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:56 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/988880978' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-24T03:00:57.320 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:56 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: ignoring --setuser ceph since I am not root 2024-03-24T03:00:57.320 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:56 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: ignoring --setgroup ceph since I am not root 2024-03-24T03:00:57.320 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:56 smithi019 ceph-mgr[37333]: -- 172.21.15.19:0/3279692243 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b19a33e5a0 con 0x55b19a312800 2024-03-24T03:00:57.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.380+0000 7fde33436640 1 -- 172.21.15.17:0/271980192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c1028e0 msgr2=0x7fde2c102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:57.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.380+0000 7fde33436640 1 --2- 172.21.15.17:0/271980192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c1028e0 0x7fde2c102d60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009900 tx=0x7fde1c02bc30 comp rx=0 tx=0).stop 2024-03-24T03:00:57.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.380+0000 7fde33436640 1 -- 172.21.15.17:0/271980192 shutdown_connections 2024-03-24T03:00:57.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.380+0000 7fde33436640 1 --2- 172.21.15.17:0/271980192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde2c1032a0 0x7fde2c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.380+0000 7fde33436640 1 --2- 172.21.15.17:0/271980192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c1028e0 0x7fde2c102d60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.380+0000 7fde33436640 1 --2- 172.21.15.17:0/271980192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2c1016f0 0x7fde2c101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.380+0000 7fde33436640 1 -- 172.21.15.17:0/271980192 >> 172.21.15.17:0/271980192 conn(0x7fde2c0fcc60 msgr2=0x7fde2c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:57.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.380+0000 7fde33436640 1 -- 172.21.15.17:0/271980192 shutdown_connections 2024-03-24T03:00:57.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.380+0000 7fde33436640 1 -- 172.21.15.17:0/271980192 wait complete. 2024-03-24T03:00:57.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde33436640 1 Processor -- start 2024-03-24T03:00:57.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde33436640 1 -- start start 2024-03-24T03:00:57.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde33436640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2c1016f0 0x7fde2c197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:57.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde33436640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde2c1028e0 0x7fde2c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:57.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde33436640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c1032a0 0x7fde2c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:57.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde33436640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c10d370 con 0x7fde2c1032a0 2024-03-24T03:00:57.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde33436640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c10d1f0 con 0x7fde2c1028e0 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde33436640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c10d4f0 con 0x7fde2c1016f0 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde319d3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c1032a0 0x7fde2c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde311d2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2c1016f0 0x7fde2c197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde309d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde2c1028e0 0x7fde2c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde319d3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c1032a0 0x7fde2c19ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55140/0 (socket says 172.21.15.17:55140) 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde311d2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2c1016f0 0x7fde2c197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.17:37158/0 (socket says 172.21.15.17:37158) 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde319d3640 1 -- 172.21.15.17:0/700225585 learned_addr learned my addr 172.21.15.17:0/700225585 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde309d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde2c1028e0 0x7fde2c197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:40930/0 (socket says 172.21.15.17:40930) 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde319d3640 1 -- 172.21.15.17:0/700225585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2c1016f0 msgr2=0x7fde2c197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde319d3640 1 --2- 172.21.15.17:0/700225585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2c1016f0 0x7fde2c197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde319d3640 1 -- 172.21.15.17:0/700225585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde2c1028e0 msgr2=0x7fde2c197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde319d3640 1 --2- 172.21.15.17:0/700225585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde2c1028e0 0x7fde2c197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.381+0000 7fde319d3640 1 -- 172.21.15.17:0/700225585 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c19f570 con 0x7fde2c1032a0 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.382+0000 7fde319d3640 1 --2- 172.21.15.17:0/700225585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c1032a0 0x7fde2c19ef00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fde2400eb10 tx=0x7fde2400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.382+0000 7fde227fc640 1 -- 172.21.15.17:0/700225585 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24004b20 con 0x7fde2c1032a0 2024-03-24T03:00:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.382+0000 7fde227fc640 1 -- 172.21.15.17:0/700225585 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde24004cc0 con 0x7fde2c1032a0 2024-03-24T03:00:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.382+0000 7fde33436640 1 -- 172.21.15.17:0/700225585 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde2c19f860 con 0x7fde2c1032a0 2024-03-24T03:00:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.382+0000 7fde227fc640 1 -- 172.21.15.17:0/700225585 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24010760 con 0x7fde2c1032a0 2024-03-24T03:00:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.382+0000 7fde33436640 1 -- 172.21.15.17:0/700225585 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2c19fd80 con 0x7fde2c1032a0 2024-03-24T03:00:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.383+0000 7fde33436640 1 -- 172.21.15.17:0/700225585 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2c1089b0 con 0x7fde2c1032a0 2024-03-24T03:00:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.384+0000 7fde227fc640 1 -- 172.21.15.17:0/700225585 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 137307+0+0 (secure 0 0 0) 0x7fde24005020 con 0x7fde2c1032a0 2024-03-24T03:00:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.384+0000 7fde227fc640 1 --2- 172.21.15.17:0/700225585 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fde180a2970 0x7fde180a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.384+0000 7fde311d2640 1 -- 172.21.15.17:0/700225585 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fde180a2970 msgr2=0x7fde180a4e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2663792883 2024-03-24T03:00:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.384+0000 7fde227fc640 1 -- 172.21.15.17:0/700225585 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fde240cf3e0 con 0x7fde2c1032a0 2024-03-24T03:00:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.384+0000 7fde311d2640 1 --2- 172.21.15.17:0/700225585 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fde180a2970 0x7fde180a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T03:00:57.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.389+0000 7fde227fc640 1 -- 172.21.15.17:0/700225585 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fde2409c040 con 0x7fde2c1032a0 2024-03-24T03:00:57.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.577+0000 7fde33436640 1 -- 172.21.15.17:0/700225585 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fde2c062310 con 0x7fde2c1032a0 2024-03-24T03:00:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.582+0000 7fde227fc640 1 -- 172.21.15.17:0/700225585 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) v1 ==== 133+0+0 (secure 0 0 0) 0x7fde2409b940 con 0x7fde2c1032a0 2024-03-24T03:00:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.584+0000 7fde33436640 1 -- 172.21.15.17:0/700225585 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fde180a2970 msgr2=0x7fde180a4e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T03:00:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.584+0000 7fde33436640 1 --2- 172.21.15.17:0/700225585 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fde180a2970 0x7fde180a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.584+0000 7fde33436640 1 -- 172.21.15.17:0/700225585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c1032a0 msgr2=0x7fde2c19ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.584+0000 7fde33436640 1 --2- 172.21.15.17:0/700225585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c1032a0 0x7fde2c19ef00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fde2400eb10 tx=0x7fde2400c2d0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.584+0000 7fde33436640 1 -- 172.21.15.17:0/700225585 shutdown_connections 2024-03-24T03:00:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.584+0000 7fde33436640 1 --2- 172.21.15.17:0/700225585 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fde180a2970 0x7fde180a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.584+0000 7fde33436640 1 --2- 172.21.15.17:0/700225585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c1032a0 0x7fde2c19ef00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.584+0000 7fde33436640 1 --2- 172.21.15.17:0/700225585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde2c1028e0 0x7fde2c197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.584+0000 7fde33436640 1 --2- 172.21.15.17:0/700225585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2c1016f0 0x7fde2c197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.584+0000 7fde33436640 1 -- 172.21.15.17:0/700225585 >> 172.21.15.17:0/700225585 conn(0x7fde2c0fcc60 msgr2=0x7fde2c105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.584+0000 7fde33436640 1 -- 172.21.15.17:0/700225585 shutdown_connections 2024-03-24T03:00:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.584+0000 7fde33436640 1 -- 172.21.15.17:0/700225585 wait complete. 2024-03-24T03:00:57.595 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-24T03:00:57.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.702+0000 7fa4eed9d640 1 -- 172.21.15.17:0/555841785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e81093c0 msgr2=0x7fa4e810b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:57.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.702+0000 7fa4eed9d640 1 --2- 172.21.15.17:0/555841785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e81093c0 0x7fa4e810b850 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009ca0 tx=0x7fa4e002c180 comp rx=0 tx=0).stop 2024-03-24T03:00:57.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.702+0000 7fa4eed9d640 1 -- 172.21.15.17:0/555841785 shutdown_connections 2024-03-24T03:00:57.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.702+0000 7fa4eed9d640 1 --2- 172.21.15.17:0/555841785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e81093c0 0x7fa4e810b850 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.702+0000 7fa4eed9d640 1 --2- 172.21.15.17:0/555841785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4e8074080 0x7fa4e80744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.702+0000 7fa4eed9d640 1 --2- 172.21.15.17:0/555841785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e8075660 0x7fa4e8073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.702+0000 7fa4eed9d640 1 -- 172.21.15.17:0/555841785 >> 172.21.15.17:0/555841785 conn(0x7fa4e80fca80 msgr2=0x7fa4e80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.702+0000 7fa4eed9d640 1 -- 172.21.15.17:0/555841785 shutdown_connections 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.702+0000 7fa4eed9d640 1 -- 172.21.15.17:0/555841785 wait complete. 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4eed9d640 1 Processor -- start 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4eed9d640 1 -- start start 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4eed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4e8074080 0x7fa4e81999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4eed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8075660 0x7fa4e8199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e81093c0 0x7fa4e81a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4eed9d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e810f3a0 con 0x7fa4e8075660 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4eed9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e810f220 con 0x7fa4e81093c0 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4eed9d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e810f520 con 0x7fa4e8074080 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4ed59a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8075660 0x7fa4e8199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4ed59a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8075660 0x7fa4e8199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55166/0 (socket says 172.21.15.17:55166) 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4ed59a640 1 -- 172.21.15.17:0/4091013371 learned_addr learned my addr 172.21.15.17:0/4091013371 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4edd9b640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4e8074080 0x7fa4e81999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4ee59c640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e81093c0 0x7fa4e81a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4ed59a640 1 -- 172.21.15.17:0/4091013371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4e8074080 msgr2=0x7fa4e81999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4ed59a640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4e8074080 0x7fa4e81999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4ed59a640 1 -- 172.21.15.17:0/4091013371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e81093c0 msgr2=0x7fa4e81a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4ed59a640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e81093c0 0x7fa4e81a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.703+0000 7fa4ed59a640 1 -- 172.21.15.17:0/4091013371 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e81a15d0 con 0x7fa4e8075660 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.704+0000 7fa4ed59a640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8075660 0x7fa4e8199ef0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400e9f0 tx=0x7fa4e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.704+0000 7fa4deffd640 1 -- 172.21.15.17:0/4091013371 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400cd80 con 0x7fa4e8075660 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.704+0000 7fa4eed9d640 1 -- 172.21.15.17:0/4091013371 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e81a18c0 con 0x7fa4e8075660 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.704+0000 7fa4deffd640 1 -- 172.21.15.17:0/4091013371 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4e4005500 con 0x7fa4e8075660 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.704+0000 7fa4deffd640 1 -- 172.21.15.17:0/4091013371 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4010690 con 0x7fa4e8075660 2024-03-24T03:00:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.704+0000 7fa4eed9d640 1 -- 172.21.15.17:0/4091013371 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e81a1e10 con 0x7fa4e8075660 2024-03-24T03:00:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.705+0000 7fa4eed9d640 1 -- 172.21.15.17:0/4091013371 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e8108180 con 0x7fa4e8075660 2024-03-24T03:00:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.706+0000 7fa4deffd640 1 -- 172.21.15.17:0/4091013371 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 137307+0+0 (secure 0 0 0) 0x7fa4e40046c0 con 0x7fa4e8075660 2024-03-24T03:00:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.707+0000 7fa4deffd640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 0x7fa4d40a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:00:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.707+0000 7fa4edd9b640 1 -- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 msgr2=0x7fa4d40a4d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2663792883 2024-03-24T03:00:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.707+0000 7fa4deffd640 1 -- 172.21.15.17:0/4091013371 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9319+0+0 (secure 0 0 0) 0x7fa4e40cf2d0 con 0x7fa4e8075660 2024-03-24T03:00:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.708+0000 7fa4edd9b640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 0x7fa4d40a4d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T03:00:57.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.713+0000 7fa4deffd640 1 -- 172.21.15.17:0/4091013371 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa4e409bf30 con 0x7fa4e8075660 2024-03-24T03:00:57.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.909+0000 7fa4edd9b640 1 -- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 msgr2=0x7fa4d40a4d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2663792883 2024-03-24T03:00:57.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.909+0000 7fa4edd9b640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 0x7fa4d40a4d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T03:00:57.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:57.921+0000 7fa4eed9d640 1 -- 172.21.15.17:0/4091013371 --> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4e8061ae0 con 0x7fa4d40a2840 2024-03-24T03:00:57.933 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:57.533+0000 7f007bc04200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T03:00:57.933 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:57.635+0000 7f007bc04200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T03:00:57.934 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:57 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:00:57.628+0000 7f0926c75200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T03:00:57.934 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:57 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:00:57.722+0000 7f0926c75200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T03:00:57.936 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:57 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:00:57.613+0000 7f02be4bb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T03:00:57.936 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:57 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:00:57.710+0000 7f02be4bb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T03:00:58.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:57 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/988880978' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-24T03:00:58.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:57 smithi017 ceph-mon[31926]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-24T03:00:58.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:57 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1371620206' entity='client.admin' 2024-03-24T03:00:58.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:00:57 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/700225585' entity='client.admin' 2024-03-24T03:00:58.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:57 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/988880978' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-24T03:00:58.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:57 smithi188 ceph-mon[36206]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-24T03:00:58.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:57 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1371620206' entity='client.admin' 2024-03-24T03:00:58.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:00:57 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/700225585' entity='client.admin' 2024-03-24T03:00:58.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:58.310+0000 7fa4edd9b640 1 -- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 msgr2=0x7fa4d40a4d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2663792883 2024-03-24T03:00:58.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:58.310+0000 7fa4edd9b640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 0x7fa4d40a4d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-24T03:00:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:57 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/988880978' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-24T03:00:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:57 smithi019 ceph-mon[35944]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-24T03:00:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:57 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1371620206' entity='client.admin' 2024-03-24T03:00:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:00:57 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/700225585' entity='client.admin' 2024-03-24T03:00:58.651 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:58 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:58.309+0000 7f007bc04200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T03:00:58.651 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:58 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:58.406+0000 7f007bc04200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T03:00:58.651 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:58 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:58.551+0000 7f007bc04200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T03:00:58.651 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:58 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:58.650+0000 7f007bc04200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T03:00:58.700 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:00:58.421+0000 7f02be4bb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T03:00:58.700 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:00:58.517+0000 7f02be4bb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T03:00:58.700 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:00:58.700+0000 7f02be4bb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T03:00:58.715 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:58 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:00:58.380+0000 7f0926c75200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T03:00:58.715 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:58 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:00:58.474+0000 7f0926c75200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T03:00:58.715 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:58 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:00:58.618+0000 7f0926c75200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T03:00:58.715 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:58 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:00:58.715+0000 7f0926c75200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T03:00:58.956 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:58 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:58.737+0000 7f007bc04200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T03:00:59.047 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:58 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:00:58.799+0000 7f0926c75200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T03:00:59.070 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:00:58.799+0000 7f02be4bb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T03:00:59.070 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:00:58.885+0000 7f02be4bb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T03:00:59.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:59.111+0000 7fa4edd9b640 1 -- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 msgr2=0x7fa4d40a4d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2663792883 2024-03-24T03:00:59.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:00:59.111+0000 7fa4edd9b640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 0x7fa4d40a4d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-24T03:00:59.312 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:59 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:59.008+0000 7f007bc04200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T03:00:59.312 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:59 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:59.122+0000 7f007bc04200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T03:00:59.312 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:59 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:59.212+0000 7f007bc04200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T03:00:59.312 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:59 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:59.312+0000 7f007bc04200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T03:00:59.445 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:00:59.184+0000 7f02be4bb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T03:00:59.445 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:00:59.344+0000 7f02be4bb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T03:00:59.473 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:00:59.110+0000 7f0926c75200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T03:00:59.473 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:00:59.228+0000 7f0926c75200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T03:00:59.473 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:00:59.335+0000 7f0926c75200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T03:00:59.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:00:59 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:00:59.492+0000 7f007bc04200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T03:00:59.796 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:00:59.445+0000 7f02be4bb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T03:00:59.796 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:00:59.567+0000 7f02be4bb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T03:00:59.797 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:00:59.473+0000 7f0926c75200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T03:00:59.797 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:00:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:00:59.664+0000 7f0926c75200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T03:01:00.070 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:00:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:00:59.796+0000 7f02be4bb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T03:01:00.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:00.712+0000 7fa4edd9b640 1 -- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 msgr2=0x7fa4d40a4d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2663792883 2024-03-24T03:01:00.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:00.712+0000 7fa4edd9b640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 0x7fa4d40a4d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-24T03:01:00.956 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:00.497+0000 7f007bc04200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T03:01:00.956 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:00.634+0000 7f007bc04200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T03:01:01.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:00 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:00.855+0000 7f0926c75200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T03:01:01.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:01 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:01.040+0000 7f0926c75200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T03:01:01.281 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:00.979+0000 7f007bc04200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T03:01:01.281 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:01 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:01.074+0000 7f007bc04200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T03:01:01.281 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:01 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:01.280+0000 7f007bc04200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T03:01:01.320 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:00 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:00.886+0000 7f02be4bb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T03:01:01.320 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:01 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:01.075+0000 7f02be4bb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T03:01:01.547 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:01 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:01.423+0000 7f0926c75200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T03:01:01.548 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:01 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:01.513+0000 7f0926c75200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T03:01:01.554 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:01 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:01.479+0000 7f007bc04200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T03:01:01.820 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:01 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:01.503+0000 7f02be4bb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T03:01:01.820 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:01 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:01.640+0000 7f02be4bb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T03:01:01.956 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:01 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:01.792+0000 7f007bc04200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T03:01:02.047 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:01 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:01.712+0000 7f0926c75200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T03:01:02.048 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:01 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:01.938+0000 7f0926c75200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T03:01:02.315 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:02 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:02.062+0000 7f007bc04200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T03:01:02.320 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:01 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:01.846+0000 7f02be4bb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T03:01:02.320 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:02.039+0000 7f02be4bb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T03:01:02.625 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:02 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:02.298+0000 7f0926c75200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T03:01:02.625 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:02 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:02.625+0000 7f0926c75200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T03:01:02.677 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:02.408+0000 7f02be4bb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T03:01:02.678 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:02.677+0000 7f02be4bb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T03:01:02.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:02 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:02.314+0000 7f007bc04200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T03:01:03.297 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:02 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:02.881+0000 7f0926c75200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T03:01:03.320 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:02.954+0000 7f02be4bb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T03:01:03.446 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:03 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:03.170+0000 7f007bc04200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T03:01:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:03.916+0000 7fa4edd9b640 1 -- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 msgr2=0x7fa4d40a4d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2663792883 2024-03-24T03:01:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:03.916+0000 7fa4edd9b640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 0x7fa4d40a4d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-24T03:01:04.094 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:03 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:03.760+0000 7f007bc04200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T03:01:04.094 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:03 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:03.863+0000 7f007bc04200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T03:01:04.094 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:04.093+0000 7f007bc04200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T03:01:04.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:04.118+0000 7fa4deffd640 1 -- 172.21.15.17:0/4091013371 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 20) v1 ==== 136975+0+0 (secure 0 0 0) 0x7fa4e4097ea0 con 0x7fa4e8075660 2024-03-24T03:01:04.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:04.118+0000 7fa4deffd640 1 -- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 msgr2=0x7fa4d40a4d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T03:01:04.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:04.118+0000 7fa4deffd640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/2663792883,v1:172.21.15.17:6801/2663792883] conn(0x7fa4d40a2840 0x7fa4d40a4d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:04.161 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:03 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:03.859+0000 7f02be4bb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T03:01:04.297 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:03 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:03.998+0000 7f0926c75200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T03:01:04.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: Active manager daemon x restarted 2024-03-24T03:01:04.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: Activating manager daemon x 2024-03-24T03:01:04.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: osdmap e94: 12 total, 12 up, 12 in 2024-03-24T03:01:04.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: mgrmap e20: x(active, starting, since 0.0117612s), standbys: z, y 2024-03-24T03:01:04.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-24T03:01:04.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-24T03:01:04.300 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T03:01:04.300 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-24T03:01:04.300 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T03:01:04.300 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T03:01:04.300 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T03:01:04.300 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T03:01:04.300 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T03:01:04.300 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T03:01:04.300 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T03:01:04.300 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T03:01:04.301 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T03:01:04.301 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T03:01:04.301 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T03:01:04.301 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T03:01:04.301 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T03:01:04.301 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T03:01:04.301 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T03:01:04.301 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T03:01:04.301 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-mon[36206]: Manager daemon x is now available 2024-03-24T03:01:04.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: Active manager daemon x restarted 2024-03-24T03:01:04.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: Activating manager daemon x 2024-03-24T03:01:04.358 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: osdmap e94: 12 total, 12 up, 12 in 2024-03-24T03:01:04.362 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: mgrmap e20: x(active, starting, since 0.0117612s), standbys: z, y 2024-03-24T03:01:04.362 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T03:01:04.362 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T03:01:04.362 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T03:01:04.362 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-24T03:01:04.362 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-24T03:01:04.362 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-24T03:01:04.362 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T03:01:04.363 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T03:01:04.364 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-mon[31926]: Manager daemon x is now available 2024-03-24T03:01:04.421 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: Active manager daemon x restarted 2024-03-24T03:01:04.422 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: Activating manager daemon x 2024-03-24T03:01:04.422 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: osdmap e94: 12 total, 12 up, 12 in 2024-03-24T03:01:04.422 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: mgrmap e20: x(active, starting, since 0.0117612s), standbys: z, y 2024-03-24T03:01:04.422 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T03:01:04.422 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T03:01:04.422 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T03:01:04.423 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-24T03:01:04.423 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-24T03:01:04.423 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-24T03:01:04.423 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-24T03:01:04.423 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-24T03:01:04.423 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-24T03:01:04.423 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-24T03:01:04.423 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-24T03:01:04.424 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-24T03:01:04.424 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-24T03:01:04.424 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-24T03:01:04.424 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-24T03:01:04.424 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T03:01:04.424 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-24T03:01:04.424 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T03:01:04.425 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T03:01:04.425 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T03:01:04.425 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T03:01:04.425 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T03:01:04.425 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T03:01:04.425 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T03:01:04.425 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T03:01:04.425 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T03:01:04.426 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-24T03:01:04.426 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-24T03:01:04.426 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-24T03:01:04.426 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-24T03:01:04.426 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T03:01:04.426 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T03:01:04.426 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T03:01:04.426 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-mon[35944]: Manager daemon x is now available 2024-03-24T03:01:04.688 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:04.542+0000 7f02be4bb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T03:01:04.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:04.356+0000 7effde599700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:01:04.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:04.356+0000 7effde599700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:01:04.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:04.356+0000 7effde599700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:01:04.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:04.356+0000 7effde599700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:01:04.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:04.356+0000 7effde599700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:01:04.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:04.356+0000 7effdb593700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:01:04.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:04.357+0000 7effdb593700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:01:04.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:04.357+0000 7effdb593700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:01:04.706 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:04.357+0000 7effdb593700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:01:04.707 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:01:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:01:04.357+0000 7effdb593700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:01:05.009 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:04 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:04.688+0000 7f02be4bb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T03:01:05.048 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:04.774+0000 7f0926c75200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T03:01:05.048 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:04 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:04.906+0000 7f0926c75200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T03:01:05.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.132+0000 7fa4deffd640 1 -- 172.21.15.17:0/4091013371 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mgrmap(e 21) v1 ==== 137102+0+0 (secure 0 0 0) 0x7fa4e409ba90 con 0x7fa4e8075660 2024-03-24T03:01:05.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.133+0000 7fa4deffd640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fa4d4051640 0x7fa4d4051a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:05.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.133+0000 7fa4deffd640 1 -- 172.21.15.17:0/4091013371 --> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4d400fde0 con 0x7fa4d4051640 2024-03-24T03:01:05.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.137+0000 7fa4edd9b640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fa4d4051640 0x7fa4d4051a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:05.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.140+0000 7fa4edd9b640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fa4d4051640 0x7fa4d4051a20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80094e0 tx=0x7fa4d8005f90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:01:05.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.216+0000 7fa4deffd640 1 -- 172.21.15.17:0/4091013371 <== mgr.25334 v2:172.21.15.17:6800/1006945247 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa4d400fde0 con 0x7fa4d4051640 2024-03-24T03:01:05.216 INFO:teuthology.orchestra.run.smithi017.stdout:Schedule set for path / 2024-03-24T03:01:05.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.218+0000 7fa4eed9d640 1 -- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fa4d4051640 msgr2=0x7fa4d4051a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.218+0000 7fa4eed9d640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fa4d4051640 0x7fa4d4051a20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80094e0 tx=0x7fa4d8005f90 comp rx=0 tx=0).stop 2024-03-24T03:01:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.218+0000 7fa4eed9d640 1 -- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8075660 msgr2=0x7fa4e8199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.218+0000 7fa4eed9d640 1 --2- 172.21.15.17:0/4091013371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8075660 0x7fa4e8199ef0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400e9f0 tx=0x7fa4e400edf0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.218+0000 7fa4ee59c640 1 -- 172.21.15.17:0/4091013371 reap_dead start 2024-03-24T03:01:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.218+0000 7fa4eed9d640 1 -- 172.21.15.17:0/4091013371 shutdown_connections 2024-03-24T03:01:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.218+0000 7fa4eed9d640 1 -- 172.21.15.17:0/4091013371 >> 172.21.15.17:0/4091013371 conn(0x7fa4e80fca80 msgr2=0x7fa4e8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:01:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.218+0000 7fa4eed9d640 1 -- 172.21.15.17:0/4091013371 shutdown_connections 2024-03-24T03:01:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.218+0000 7fa4eed9d640 1 -- 172.21.15.17:0/4091013371 wait complete. 2024-03-24T03:01:05.229 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-24T03:01:05.320 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:01:05 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y[37329]: 2024-03-24T03:01:05.009+0000 7f02be4bb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T03:01:05.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:05 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:01:05.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:05 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:01:05.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:05 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-24T03:01:05.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:05 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-24T03:01:05.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:05 smithi019 ceph-mon[35944]: Standby manager daemon y restarted 2024-03-24T03:01:05.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:05 smithi019 ceph-mon[35944]: Standby manager daemon y started 2024-03-24T03:01:05.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:05 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.19:0/3614921410' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T03:01:05.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:05 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.19:0/3614921410' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T03:01:05.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:05 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.19:0/3614921410' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T03:01:05.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:05 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.19:0/3614921410' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T03:01:05.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:05 smithi019 ceph-mon[35944]: mgrmap e21: x(active, since 1.02755s), standbys: y, z 2024-03-24T03:01:05.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.337+0000 7f33b3af8640 1 -- 172.21.15.17:0/3750531898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1005c0 msgr2=0x7f33ac1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:05.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.337+0000 7f33b3af8640 1 --2- 172.21.15.17:0/3750531898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1005c0 0x7f33ac1009e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009900 tx=0x7f33a402bc30 comp rx=0 tx=0).stop 2024-03-24T03:01:05.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.337+0000 7f33b3af8640 1 -- 172.21.15.17:0/3750531898 shutdown_connections 2024-03-24T03:01:05.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.337+0000 7f33b3af8640 1 --2- 172.21.15.17:0/3750531898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33ac102170 0x7f33ac108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.337+0000 7f33b3af8640 1 --2- 172.21.15.17:0/3750531898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac1017b0 0x7f33ac101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.337+0000 7f33b3af8640 1 --2- 172.21.15.17:0/3750531898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1005c0 0x7f33ac1009e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.337+0000 7f33b3af8640 1 -- 172.21.15.17:0/3750531898 >> 172.21.15.17:0/3750531898 conn(0x7f33ac0fbb30 msgr2=0x7f33ac0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.337+0000 7f33b3af8640 1 -- 172.21.15.17:0/3750531898 shutdown_connections 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.337+0000 7f33b3af8640 1 -- 172.21.15.17:0/3750531898 wait complete. 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b3af8640 1 Processor -- start 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b3af8640 1 -- start start 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b3af8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac1005c0 0x7f33ac197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b3af8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1017b0 0x7f33ac197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b3af8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33ac102170 0x7f33ac19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b3af8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac10c230 con 0x7f33ac1017b0 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b3af8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac10c0b0 con 0x7f33ac1005c0 2024-03-24T03:01:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b3af8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac10c3b0 con 0x7f33ac102170 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b1093640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1017b0 0x7f33ac197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b1093640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1017b0 0x7f33ac197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35960/0 (socket says 172.21.15.17:35960) 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b1093640 1 -- 172.21.15.17:0/4084089222 learned_addr learned my addr 172.21.15.17:0/4084089222 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b1894640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac1005c0 0x7f33ac197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b2095640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33ac102170 0x7f33ac19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b1093640 1 -- 172.21.15.17:0/4084089222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33ac102170 msgr2=0x7f33ac19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b1093640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33ac102170 0x7f33ac19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b1093640 1 -- 172.21.15.17:0/4084089222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac1005c0 msgr2=0x7f33ac197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b1093640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac1005c0 0x7f33ac197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b1093640 1 -- 172.21.15.17:0/4084089222 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac19f580 con 0x7f33ac1017b0 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.338+0000 7f33b1093640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1017b0 0x7f33ac197ea0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f33a000e9f0 tx=0x7f33a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.339+0000 7f33aaffd640 1 -- 172.21.15.17:0/4084089222 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a000cd80 con 0x7f33ac1017b0 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.339+0000 7f33b3af8640 1 -- 172.21.15.17:0/4084089222 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac19f870 con 0x7f33ac1017b0 2024-03-24T03:01:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.339+0000 7f33aaffd640 1 -- 172.21.15.17:0/4084089222 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33a0010910 con 0x7f33ac1017b0 2024-03-24T03:01:05.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.339+0000 7f33aaffd640 1 -- 172.21.15.17:0/4084089222 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a00188f0 con 0x7f33ac1017b0 2024-03-24T03:01:05.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.339+0000 7f33b3af8640 1 -- 172.21.15.17:0/4084089222 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac19fdc0 con 0x7f33ac1017b0 2024-03-24T03:01:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.340+0000 7f33b3af8640 1 -- 172.21.15.17:0/4084089222 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac19ffc0 con 0x7f33ac1017b0 2024-03-24T03:01:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.341+0000 7f33aaffd640 1 -- 172.21.15.17:0/4084089222 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 21) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f33a0010ac0 con 0x7f33ac1017b0 2024-03-24T03:01:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.341+0000 7f33aaffd640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f33940a26b0 0x7f33940a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.341+0000 7f33b1894640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f33940a26b0 0x7f33940a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.341+0000 7f33aaffd640 1 -- 172.21.15.17:0/4084089222 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9577+0+0 (secure 0 0 0) 0x7f33a00cef50 con 0x7f33ac1017b0 2024-03-24T03:01:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.342+0000 7f33b1894640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f33940a26b0 0x7f33940a4b70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f33a4000c00 tx=0x7f33a4005fd0 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:01:05.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.346+0000 7f33aaffd640 1 -- 172.21.15.17:0/4084089222 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f33a009b750 con 0x7f33ac1017b0 2024-03-24T03:01:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:05 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:01:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:05 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:01:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:05 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-24T03:01:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:05 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-24T03:01:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:05 smithi017 ceph-mon[31926]: Standby manager daemon y restarted 2024-03-24T03:01:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:05 smithi017 ceph-mon[31926]: Standby manager daemon y started 2024-03-24T03:01:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:05 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.19:0/3614921410' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T03:01:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:05 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.19:0/3614921410' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T03:01:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:05 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.19:0/3614921410' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T03:01:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:05 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.19:0/3614921410' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T03:01:05.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:05 smithi017 ceph-mon[31926]: mgrmap e21: x(active, since 1.02755s), standbys: y, z 2024-03-24T03:01:05.540 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:01:05 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z[38061]: 2024-03-24T03:01:05.228+0000 7f0926c75200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T03:01:05.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:05 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:01:05.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:05 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:01:05.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:05 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-24T03:01:05.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:05 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-24T03:01:05.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:05 smithi188 ceph-mon[36206]: Standby manager daemon y restarted 2024-03-24T03:01:05.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:05 smithi188 ceph-mon[36206]: Standby manager daemon y started 2024-03-24T03:01:05.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:05 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.19:0/3614921410' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T03:01:05.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:05 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.19:0/3614921410' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T03:01:05.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:05 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.19:0/3614921410' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T03:01:05.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:05 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.19:0/3614921410' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T03:01:05.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:05 smithi188 ceph-mon[36206]: mgrmap e21: x(active, since 1.02755s), standbys: y, z 2024-03-24T03:01:05.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.564+0000 7f33b3af8640 1 -- 172.21.15.17:0/4084089222 --> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f33ac0fdca0 con 0x7f33940a26b0 2024-03-24T03:01:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.622+0000 7f33aaffd640 1 -- 172.21.15.17:0/4084089222 <== mgr.25334 v2:172.21.15.17:6800/1006945247 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f33ac0fdca0 con 0x7f33940a26b0 2024-03-24T03:01:05.622 INFO:teuthology.orchestra.run.smithi017.stdout:Retention added to path / 2024-03-24T03:01:05.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.623+0000 7f33b3af8640 1 -- 172.21.15.17:0/4084089222 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f33940a26b0 msgr2=0x7f33940a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:05.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.624+0000 7f33b3af8640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f33940a26b0 0x7f33940a4b70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f33a4000c00 tx=0x7f33a4005fd0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.624+0000 7f33b3af8640 1 -- 172.21.15.17:0/4084089222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1017b0 msgr2=0x7f33ac197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:05.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.624+0000 7f33b3af8640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1017b0 0x7f33ac197ea0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f33a000e9f0 tx=0x7f33a000eeb0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.624+0000 7f33b3af8640 1 -- 172.21.15.17:0/4084089222 shutdown_connections 2024-03-24T03:01:05.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.624+0000 7f33b3af8640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f33940a26b0 0x7f33940a4b70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.624+0000 7f33b3af8640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33ac102170 0x7f33ac19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.624+0000 7f33b3af8640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1017b0 0x7f33ac197ea0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.624+0000 7f33b3af8640 1 --2- 172.21.15.17:0/4084089222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33ac1005c0 0x7f33ac197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.624+0000 7f33b3af8640 1 -- 172.21.15.17:0/4084089222 >> 172.21.15.17:0/4084089222 conn(0x7f33ac0fbb30 msgr2=0x7f33ac0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:01:05.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.624+0000 7f33b3af8640 1 -- 172.21.15.17:0/4084089222 shutdown_connections 2024-03-24T03:01:05.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.624+0000 7f33b3af8640 1 -- 172.21.15.17:0/4084089222 wait complete. 2024-03-24T03:01:05.635 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-24T03:01:05.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.740+0000 7fc3fe24b640 1 -- 172.21.15.17:0/876923679 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f81005c0 msgr2=0x7fc3f81009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:05.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.740+0000 7fc3fe24b640 1 --2- 172.21.15.17:0/876923679 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f81005c0 0x7fc3f81009e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009900 tx=0x7fc3ec02bc30 comp rx=0 tx=0).stop 2024-03-24T03:01:05.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.740+0000 7fc3fe24b640 1 -- 172.21.15.17:0/876923679 shutdown_connections 2024-03-24T03:01:05.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.740+0000 7fc3fe24b640 1 --2- 172.21.15.17:0/876923679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f8102170 0x7fc3f8108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.740+0000 7fc3fe24b640 1 --2- 172.21.15.17:0/876923679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f81017b0 0x7fc3f8101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.740+0000 7fc3fe24b640 1 --2- 172.21.15.17:0/876923679 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f81005c0 0x7fc3f81009e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.740+0000 7fc3fe24b640 1 -- 172.21.15.17:0/876923679 >> 172.21.15.17:0/876923679 conn(0x7fc3f80fbb30 msgr2=0x7fc3f80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:01:05.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.740+0000 7fc3fe24b640 1 -- 172.21.15.17:0/876923679 shutdown_connections 2024-03-24T03:01:05.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.740+0000 7fc3fe24b640 1 -- 172.21.15.17:0/876923679 wait complete. 2024-03-24T03:01:05.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3fe24b640 1 Processor -- start 2024-03-24T03:01:05.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3fe24b640 1 -- start start 2024-03-24T03:01:05.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3fe24b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f81005c0 0x7fc3f8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:05.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3fe24b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f81017b0 0x7fc3f8197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:05.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3fe24b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f8102170 0x7fc3f819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:05.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3fe24b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f810c210 con 0x7fc3f8102170 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3fe24b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f810c090 con 0x7fc3f81017b0 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3fe24b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f810c390 con 0x7fc3f81005c0 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3f7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f8102170 0x7fc3f819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3f77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f81005c0 0x7fc3f8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3f7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f8102170 0x7fc3f819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35978/0 (socket says 172.21.15.17:35978) 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3f6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f81017b0 0x7fc3f8197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3f7fff640 1 -- 172.21.15.17:0/548990389 learned_addr learned my addr 172.21.15.17:0/548990389 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3f7fff640 1 -- 172.21.15.17:0/548990389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f81005c0 msgr2=0x7fc3f8197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3f7fff640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f81005c0 0x7fc3f8197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3f7fff640 1 -- 172.21.15.17:0/548990389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f81017b0 msgr2=0x7fc3f8197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3f7fff640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f81017b0 0x7fc3f8197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.741+0000 7fc3f7fff640 1 -- 172.21.15.17:0/548990389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f819f570 con 0x7fc3f8102170 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.742+0000 7fc3f6ffd640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f81017b0 0x7fc3f8197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.742+0000 7fc3f7fff640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f8102170 0x7fc3f819ef00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009fd0 tx=0x7fc3f000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:01:05.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.742+0000 7fc3f4ff9640 1 -- 172.21.15.17:0/548990389 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0017070 con 0x7fc3f8102170 2024-03-24T03:01:05.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.742+0000 7fc3f4ff9640 1 -- 172.21.15.17:0/548990389 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3f0003e50 con 0x7fc3f8102170 2024-03-24T03:01:05.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.742+0000 7fc3fe24b640 1 -- 172.21.15.17:0/548990389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f819f860 con 0x7fc3f8102170 2024-03-24T03:01:05.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.742+0000 7fc3f4ff9640 1 -- 172.21.15.17:0/548990389 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00106c0 con 0x7fc3f8102170 2024-03-24T03:01:05.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.742+0000 7fc3fe24b640 1 -- 172.21.15.17:0/548990389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f819fd60 con 0x7fc3f8102170 2024-03-24T03:01:05.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.743+0000 7fc3fe24b640 1 -- 172.21.15.17:0/548990389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f819ffc0 con 0x7fc3f8102170 2024-03-24T03:01:05.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.744+0000 7fc3f4ff9640 1 -- 172.21.15.17:0/548990389 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 21) v1 ==== 137102+0+0 (secure 0 0 0) 0x7fc3f0003970 con 0x7fc3f8102170 2024-03-24T03:01:05.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.744+0000 7fc3f4ff9640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc3e00a26b0 0x7fc3e00a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:05.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.745+0000 7fc3f77fe640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc3e00a26b0 0x7fc3e00a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:05.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.745+0000 7fc3f4ff9640 1 -- 172.21.15.17:0/548990389 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9577+0+0 (secure 0 0 0) 0x7fc3f00ce880 con 0x7fc3f8102170 2024-03-24T03:01:05.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.745+0000 7fc3f77fe640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc3e00a26b0 0x7fc3e00a4b70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec000c00 tx=0x7fc3ec005fd0 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:01:05.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.752+0000 7fc3f4ff9640 1 -- 172.21.15.17:0/548990389 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc3f009b080 con 0x7fc3f8102170 2024-03-24T03:01:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.963+0000 7fc3fe24b640 1 -- 172.21.15.17:0/548990389 --> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3f80fdcc0 con 0x7fc3e00a26b0 2024-03-24T03:01:05.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.977+0000 7fc3f4ff9640 1 -- 172.21.15.17:0/548990389 <== mgr.25334 v2:172.21.15.17:6800/1006945247 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7fc3f80fdcc0 con 0x7fc3e00a26b0 2024-03-24T03:01:05.977 INFO:teuthology.orchestra.run.smithi017.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-24T00:00:00", "created": "2024-03-24T03:01:05", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-24T03:01:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.979+0000 7fc3fe24b640 1 -- 172.21.15.17:0/548990389 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc3e00a26b0 msgr2=0x7fc3e00a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.979+0000 7fc3fe24b640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc3e00a26b0 0x7fc3e00a4b70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec000c00 tx=0x7fc3ec005fd0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.979+0000 7fc3fe24b640 1 -- 172.21.15.17:0/548990389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f8102170 msgr2=0x7fc3f819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.979+0000 7fc3fe24b640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f8102170 0x7fc3f819ef00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009fd0 tx=0x7fc3f000b260 comp rx=0 tx=0).stop 2024-03-24T03:01:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.980+0000 7fc3fe24b640 1 -- 172.21.15.17:0/548990389 shutdown_connections 2024-03-24T03:01:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.980+0000 7fc3fe24b640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc3e00a26b0 0x7fc3e00a4b70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.980+0000 7fc3fe24b640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f8102170 0x7fc3f819ef00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.980+0000 7fc3fe24b640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f81017b0 0x7fc3f8197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.980+0000 7fc3fe24b640 1 --2- 172.21.15.17:0/548990389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f81005c0 0x7fc3f8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.980+0000 7fc3fe24b640 1 -- 172.21.15.17:0/548990389 >> 172.21.15.17:0/548990389 conn(0x7fc3f80fbb30 msgr2=0x7fc3f80fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:01:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.980+0000 7fc3fe24b640 1 -- 172.21.15.17:0/548990389 shutdown_connections 2024-03-24T03:01:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:05.980+0000 7fc3fe24b640 1 -- 172.21.15.17:0/548990389 wait complete. 2024-03-24T03:01:05.992 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-24T03:01:06.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.113+0000 7fca01ba9640 1 -- 172.21.15.17:0/4011540919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc1028c0 msgr2=0x7fc9fc102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:06.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.113+0000 7fca01ba9640 1 --2- 172.21.15.17:0/4011540919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc1028c0 0x7fc9fc102d40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009900 tx=0x7fc9f002bc30 comp rx=0 tx=0).stop 2024-03-24T03:01:06.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.113+0000 7fca01ba9640 1 -- 172.21.15.17:0/4011540919 shutdown_connections 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.113+0000 7fca01ba9640 1 --2- 172.21.15.17:0/4011540919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9fc103280 0x7fc9fc109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.113+0000 7fca01ba9640 1 --2- 172.21.15.17:0/4011540919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc1028c0 0x7fc9fc102d40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.113+0000 7fca01ba9640 1 --2- 172.21.15.17:0/4011540919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9fc1016d0 0x7fc9fc101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.113+0000 7fca01ba9640 1 -- 172.21.15.17:0/4011540919 >> 172.21.15.17:0/4011540919 conn(0x7fc9fc0fcc60 msgr2=0x7fc9fc0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.113+0000 7fca01ba9640 1 -- 172.21.15.17:0/4011540919 shutdown_connections 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.113+0000 7fca01ba9640 1 -- 172.21.15.17:0/4011540919 wait complete. 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.113+0000 7fca01ba9640 1 Processor -- start 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.113+0000 7fca01ba9640 1 -- start start 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fca01ba9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9fc1016d0 0x7fc9fc197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fca01ba9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc1028c0 0x7fc9fc197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fca01ba9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9fc103280 0x7fc9fc19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fca01ba9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc10d350 con 0x7fc9fc1028c0 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fca01ba9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc10d1d0 con 0x7fc9fc1016d0 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fca01ba9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc10d4d0 con 0x7fc9fc103280 2024-03-24T03:01:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fc9faffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc1028c0 0x7fc9fc197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fc9faffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc1028c0 0x7fc9fc197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36004/0 (socket says 172.21.15.17:36004) 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fc9faffd640 1 -- 172.21.15.17:0/666401617 learned_addr learned my addr 172.21.15.17:0/666401617 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fc9fb7fe640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9fc1016d0 0x7fc9fc197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fc9fbfff640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9fc103280 0x7fc9fc19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fc9faffd640 1 -- 172.21.15.17:0/666401617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9fc103280 msgr2=0x7fc9fc19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fc9faffd640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9fc103280 0x7fc9fc19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fc9faffd640 1 -- 172.21.15.17:0/666401617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9fc1016d0 msgr2=0x7fc9fc197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fc9faffd640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9fc1016d0 0x7fc9fc197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fc9faffd640 1 -- 172.21.15.17:0/666401617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc19f550 con 0x7fc9fc1028c0 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.114+0000 7fc9faffd640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc1028c0 0x7fc9fc197e70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009230 tx=0x7fc9f002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.115+0000 7fc9f8ff9640 1 -- 172.21.15.17:0/666401617 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0041070 con 0x7fc9fc1028c0 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.115+0000 7fca01ba9640 1 -- 172.21.15.17:0/666401617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9fc19f7e0 con 0x7fc9fc1028c0 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.115+0000 7fc9f8ff9640 1 -- 172.21.15.17:0/666401617 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9f0004450 con 0x7fc9fc1028c0 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.115+0000 7fc9f8ff9640 1 -- 172.21.15.17:0/666401617 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0003e20 con 0x7fc9fc1028c0 2024-03-24T03:01:06.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.115+0000 7fca01ba9640 1 -- 172.21.15.17:0/666401617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9fc19fcd0 con 0x7fc9fc1028c0 2024-03-24T03:01:06.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.116+0000 7fca01ba9640 1 -- 172.21.15.17:0/666401617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9fc1089f0 con 0x7fc9fc1028c0 2024-03-24T03:01:06.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.117+0000 7fc9f8ff9640 1 -- 172.21.15.17:0/666401617 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 21) v1 ==== 137102+0+0 (secure 0 0 0) 0x7fc9f0044070 con 0x7fc9fc1028c0 2024-03-24T03:01:06.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.117+0000 7fc9f8ff9640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc9e40a26b0 0x7fc9e40a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:01:06.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.118+0000 7fc9f8ff9640 1 -- 172.21.15.17:0/666401617 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9577+0+0 (secure 0 0 0) 0x7fc9f00f0730 con 0x7fc9fc1028c0 2024-03-24T03:01:06.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.118+0000 7fc9fb7fe640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc9e40a26b0 0x7fc9e40a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:01:06.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.118+0000 7fc9fb7fe640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc9e40a26b0 0x7fc9e40a4b70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc102720 tx=0x7fc9ec009500 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:01:06.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.122+0000 7fc9f8ff9640 1 -- 172.21.15.17:0/666401617 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc9f00bcf30 con 0x7fc9fc1028c0 2024-03-24T03:01:06.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.184+0000 7fc9f8ff9640 1 -- 172.21.15.17:0/666401617 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 22) v1 ==== 137410+0+0 (secure 0 0 0) 0x7fc9f00035f0 con 0x7fc9fc1028c0 2024-03-24T03:01:06.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.372+0000 7fca01ba9640 1 -- 172.21.15.17:0/666401617 --> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc9fc061ae0 con 0x7fc9e40a26b0 2024-03-24T03:01:06.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.379+0000 7fc9f8ff9640 1 -- 172.21.15.17:0/666401617 <== mgr.25334 v2:172.21.15.17:6800/1006945247 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fc9fc061ae0 con 0x7fc9e40a26b0 2024-03-24T03:01:06.379 INFO:teuthology.orchestra.run.smithi017.stdout:/ 1m 6m3h 2024-03-24T03:01:06.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.381+0000 7fca01ba9640 1 -- 172.21.15.17:0/666401617 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc9e40a26b0 msgr2=0x7fc9e40a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:06.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.381+0000 7fca01ba9640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc9e40a26b0 0x7fc9e40a4b70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc102720 tx=0x7fc9ec009500 comp rx=0 tx=0).stop 2024-03-24T03:01:06.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.381+0000 7fca01ba9640 1 -- 172.21.15.17:0/666401617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc1028c0 msgr2=0x7fc9fc197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:01:06.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.381+0000 7fca01ba9640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc1028c0 0x7fc9fc197e70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009230 tx=0x7fc9f002c650 comp rx=0 tx=0).stop 2024-03-24T03:01:06.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.381+0000 7fca01ba9640 1 -- 172.21.15.17:0/666401617 shutdown_connections 2024-03-24T03:01:06.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.381+0000 7fca01ba9640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc9e40a26b0 0x7fc9e40a4b70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:06.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.381+0000 7fca01ba9640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9fc103280 0x7fc9fc19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:06.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.381+0000 7fca01ba9640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc1028c0 0x7fc9fc197e70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:06.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.381+0000 7fca01ba9640 1 --2- 172.21.15.17:0/666401617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9fc1016d0 0x7fc9fc197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:01:06.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.381+0000 7fca01ba9640 1 -- 172.21.15.17:0/666401617 >> 172.21.15.17:0/666401617 conn(0x7fc9fc0fcc60 msgr2=0x7fc9fc105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:01:06.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.382+0000 7fca01ba9640 1 -- 172.21.15.17:0/666401617 shutdown_connections 2024-03-24T03:01:06.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:01:06.382+0000 7fca01ba9640 1 -- 172.21.15.17:0/666401617 wait complete. 2024-03-24T03:01:06.392 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:06 smithi017 ceph-mon[31926]: Standby manager daemon z restarted 2024-03-24T03:01:06.392 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:06 smithi017 ceph-mon[31926]: Standby manager daemon z started 2024-03-24T03:01:06.393 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:06 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.188:0/343725292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-24T03:01:06.393 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:06 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.188:0/343725292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T03:01:06.393 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:06 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.188:0/343725292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-24T03:01:06.393 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:06 smithi017 ceph-mon[31926]: from='mgr.? 172.21.15.188:0/343725292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T03:01:06.393 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:06 smithi017 ceph-mon[31926]: [24/Mar/2024:03:01:05] ENGINE Bus STARTING 2024-03-24T03:01:06.393 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:06 smithi017 ceph-mon[31926]: from='client.15642 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:01:06.393 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:06 smithi017 ceph-mon[31926]: [24/Mar/2024:03:01:05] ENGINE Serving on http://172.21.15.17:8765 2024-03-24T03:01:06.394 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-24T03:01:06.423 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-24T03:01:06.433 INFO:teuthology.run_tasks:Running task workunit... 2024-03-24T03:01:06.484 INFO:tasks.workunit:Pulling workunits from ref 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T03:01:06.484 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-24T03:01:06.484 INFO:tasks.workunit:timeout=3h 2024-03-24T03:01:06.484 INFO:tasks.workunit:cleanup=True 2024-03-24T03:01:06.485 DEBUG:teuthology.orchestra.run.smithi017:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-24T03:01:06.500 INFO:teuthology.orchestra.run.smithi017.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-24T03:01:06.500 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-24T03:01:06.500 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-24T03:01:06.500 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T03:01:06.500 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-24T03:01:06.500 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-24 02:58:59.171701689 +0000 2024-03-24T03:01:06.500 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-24 02:58:59.171701689 +0000 2024-03-24T03:01:06.500 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-24 03:00:54.317761211 +0000 2024-03-24T03:01:06.500 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-03-24 02:58:59.171701689 +0000 2024-03-24T03:01:06.501 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-24T03:01:06.501 DEBUG:teuthology.orchestra.run.smithi017:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-24T03:01:06.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:06 smithi188 ceph-mon[36206]: Standby manager daemon z restarted 2024-03-24T03:01:06.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:06 smithi188 ceph-mon[36206]: Standby manager daemon z started 2024-03-24T03:01:06.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:06 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.188:0/343725292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-24T03:01:06.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:06 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.188:0/343725292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T03:01:06.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:06 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.188:0/343725292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-24T03:01:06.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:06 smithi188 ceph-mon[36206]: from='mgr.? 172.21.15.188:0/343725292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T03:01:06.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:06 smithi188 ceph-mon[36206]: [24/Mar/2024:03:01:05] ENGINE Bus STARTING 2024-03-24T03:01:06.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:06 smithi188 ceph-mon[36206]: from='client.15642 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:01:06.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:06 smithi188 ceph-mon[36206]: [24/Mar/2024:03:01:05] ENGINE Serving on http://172.21.15.17:8765 2024-03-24T03:01:06.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:06 smithi019 ceph-mon[35944]: Standby manager daemon z restarted 2024-03-24T03:01:06.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:06 smithi019 ceph-mon[35944]: Standby manager daemon z started 2024-03-24T03:01:06.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:06 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.188:0/343725292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-24T03:01:06.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:06 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.188:0/343725292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T03:01:06.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:06 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.188:0/343725292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-24T03:01:06.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:06 smithi019 ceph-mon[35944]: from='mgr.? 172.21.15.188:0/343725292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T03:01:06.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:06 smithi019 ceph-mon[35944]: [24/Mar/2024:03:01:05] ENGINE Bus STARTING 2024-03-24T03:01:06.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:06 smithi019 ceph-mon[35944]: from='client.15642 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:01:06.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:06 smithi019 ceph-mon[35944]: [24/Mar/2024:03:01:05] ENGINE Serving on http://172.21.15.17:8765 2024-03-24T03:01:06.578 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T03:01:06.633 INFO:tasks.workunit.client.0.smithi017.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-24T03:01:07.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:07 smithi017 ceph-mon[31926]: [24/Mar/2024:03:01:06] ENGINE Serving on https://172.21.15.17:7150 2024-03-24T03:01:07.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:07 smithi017 ceph-mon[31926]: [24/Mar/2024:03:01:06] ENGINE Bus STARTED 2024-03-24T03:01:07.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:07 smithi017 ceph-mon[31926]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T03:01:07.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:07 smithi017 ceph-mon[31926]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-03-24T03:01:07.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:07 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:07 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:07 smithi017 ceph-mon[31926]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:01:07.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:07 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:07 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:07 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:07 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:07 smithi188 ceph-mon[36206]: [24/Mar/2024:03:01:06] ENGINE Serving on https://172.21.15.17:7150 2024-03-24T03:01:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:07 smithi188 ceph-mon[36206]: [24/Mar/2024:03:01:06] ENGINE Bus STARTED 2024-03-24T03:01:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:07 smithi188 ceph-mon[36206]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T03:01:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:07 smithi188 ceph-mon[36206]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-03-24T03:01:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:07 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:07 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:07 smithi188 ceph-mon[36206]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:01:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:07 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:07 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:07 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:07 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:07 smithi019 ceph-mon[35944]: [24/Mar/2024:03:01:06] ENGINE Serving on https://172.21.15.17:7150 2024-03-24T03:01:07.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:07 smithi019 ceph-mon[35944]: [24/Mar/2024:03:01:06] ENGINE Bus STARTED 2024-03-24T03:01:07.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:07 smithi019 ceph-mon[35944]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T03:01:07.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:07 smithi019 ceph-mon[35944]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-03-24T03:01:07.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:07 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:07 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:07 smithi019 ceph-mon[35944]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:01:07.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:07 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:07 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:07 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:07.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:07 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi017 to 177.2M 2024-03-24T03:01:08.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: Unable to set osd_memory_target on smithi017 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi019 to 175.2M 2024-03-24T03:01:08.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: Unable to set osd_memory_target on smithi019 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-03-24T03:01:08.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:01:08.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:01:08.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: Updating smithi017:/etc/ceph/ceph.conf 2024-03-24T03:01:08.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-24T03:01:08.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:08 smithi188 ceph-mon[36206]: Updating smithi188:/etc/ceph/ceph.conf 2024-03-24T03:01:08.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi017 to 177.2M 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: Unable to set osd_memory_target on smithi017 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi019 to 175.2M 2024-03-24T03:01:08.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: Unable to set osd_memory_target on smithi019 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-03-24T03:01:08.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:01:08.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:01:08.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: Updating smithi017:/etc/ceph/ceph.conf 2024-03-24T03:01:08.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-24T03:01:08.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:08 smithi019 ceph-mon[35944]: Updating smithi188:/etc/ceph/ceph.conf 2024-03-24T03:01:08.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi017 to 177.2M 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: Unable to set osd_memory_target on smithi017 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi019 to 175.2M 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: Unable to set osd_memory_target on smithi019 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: Updating smithi017:/etc/ceph/ceph.conf 2024-03-24T03:01:08.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-24T03:01:08.959 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:08 smithi017 ceph-mon[31926]: Updating smithi188:/etc/ceph/ceph.conf 2024-03-24T03:01:10.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: Updating smithi017:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: Updating smithi017:/etc/ceph/ceph.client.admin.keyring 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: Updating smithi017:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:01:10.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:01:10.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:01:10.208 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T03:01:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T03:01:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T03:01:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: Updating smithi017:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T03:01:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-24T03:01:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: Updating smithi017:/etc/ceph/ceph.client.admin.keyring 2024-03-24T03:01:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-03-24T03:01:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T03:01:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T03:01:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: Updating smithi017:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T03:01:10.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-24T03:01:10.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:01:10.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:01:10.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:01:10.299 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T03:01:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T03:01:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T03:01:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: Updating smithi017:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T03:01:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-24T03:01:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: Updating smithi017:/etc/ceph/ceph.client.admin.keyring 2024-03-24T03:01:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: Updating smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: Updating smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: Updating smithi017:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:01:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:01:10.322 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:01:12.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:11 smithi017 ceph-mon[31926]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 184 KiB/s wr, 80 op/s 2024-03-24T03:01:12.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:11 smithi188 ceph-mon[36206]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 184 KiB/s wr, 80 op/s 2024-03-24T03:01:12.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:11 smithi019 ceph-mon[35944]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 184 KiB/s wr, 80 op/s 2024-03-24T03:01:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:13 smithi017 ceph-mon[31926]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-24T03:01:14.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:13 smithi188 ceph-mon[36206]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-24T03:01:14.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:13 smithi019 ceph-mon[35944]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-24T03:01:16.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:15 smithi017 ceph-mon[31926]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-24T03:01:16.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:15 smithi188 ceph-mon[36206]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-24T03:01:16.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:15 smithi019 ceph-mon[35944]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-24T03:01:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:17 smithi017 ceph-mon[31926]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-24T03:01:18.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:17 smithi188 ceph-mon[36206]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-24T03:01:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:17 smithi019 ceph-mon[35944]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-24T03:01:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:19 smithi017 ceph-mon[31926]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-24T03:01:20.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:19 smithi188 ceph-mon[36206]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-24T03:01:20.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:19 smithi019 ceph-mon[35944]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-24T03:01:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:21 smithi017 ceph-mon[31926]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-03-24T03:01:22.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:21 smithi188 ceph-mon[36206]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-03-24T03:01:22.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:21 smithi019 ceph-mon[35944]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-03-24T03:01:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:23 smithi017 ceph-mon[31926]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 767 B/s wr, 21 op/s 2024-03-24T03:01:24.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:23 smithi188 ceph-mon[36206]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 767 B/s wr, 21 op/s 2024-03-24T03:01:24.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:23 smithi019 ceph-mon[35944]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 767 B/s wr, 21 op/s 2024-03-24T03:01:26.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:25 smithi017 ceph-mon[31926]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:26.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:25 smithi188 ceph-mon[36206]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:26.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:25 smithi019 ceph-mon[35944]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:28.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:27 smithi017 ceph-mon[31926]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:01:28.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:27 smithi188 ceph-mon[36206]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:01:28.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:27 smithi019 ceph-mon[35944]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:01:30.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:29 smithi017 ceph-mon[31926]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:30.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:29 smithi188 ceph-mon[36206]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:29 smithi019 ceph-mon[35944]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:32.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:31 smithi017 ceph-mon[31926]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:01:32.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:31 smithi188 ceph-mon[36206]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:01:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:31 smithi019 ceph-mon[35944]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:01:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:33 smithi017 ceph-mon[31926]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-24T03:01:34.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:33 smithi188 ceph-mon[36206]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-24T03:01:34.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:33 smithi019 ceph-mon[35944]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-24T03:01:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:35 smithi017 ceph-mon[31926]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:36.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:35 smithi188 ceph-mon[36206]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:35 smithi019 ceph-mon[35944]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:38.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:37 smithi017 ceph-mon[31926]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:01:38.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:37 smithi188 ceph-mon[36206]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:01:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:37 smithi019 ceph-mon[35944]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:01:40.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:39 smithi017 ceph-mon[31926]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:40.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:39 smithi188 ceph-mon[36206]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:39 smithi019 ceph-mon[35944]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:42.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:41 smithi017 ceph-mon[31926]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-24T03:01:42.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:41 smithi188 ceph-mon[36206]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-24T03:01:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:41 smithi019 ceph-mon[35944]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-24T03:01:44.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:43 smithi017 ceph-mon[31926]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-24T03:01:44.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:43 smithi188 ceph-mon[36206]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-24T03:01:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:43 smithi019 ceph-mon[35944]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-24T03:01:46.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:45 smithi017 ceph-mon[31926]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-24T03:01:46.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:45 smithi188 ceph-mon[36206]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-24T03:01:46.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:45 smithi019 ceph-mon[35944]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-24T03:01:48.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:47 smithi017 ceph-mon[31926]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-24T03:01:48.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:47 smithi188 ceph-mon[36206]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-24T03:01:48.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:47 smithi019 ceph-mon[35944]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-24T03:01:50.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:49 smithi017 ceph-mon[31926]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-24T03:01:50.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:49 smithi188 ceph-mon[36206]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-24T03:01:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:49 smithi019 ceph-mon[35944]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-24T03:01:52.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:51 smithi017 ceph-mon[31926]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-24T03:01:52.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:51 smithi188 ceph-mon[36206]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-24T03:01:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:51 smithi019 ceph-mon[35944]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-24T03:01:54.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:53 smithi017 ceph-mon[31926]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:01:54.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:53 smithi188 ceph-mon[36206]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:01:54.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:53 smithi019 ceph-mon[35944]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:01:56.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:55 smithi017 ceph-mon[31926]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:56.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:55 smithi188 ceph-mon[36206]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:55 smithi019 ceph-mon[35944]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:01:58.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:57 smithi017 ceph-mon[31926]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:01:58.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:57 smithi188 ceph-mon[36206]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:01:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:57 smithi019 ceph-mon[35944]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:02:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:01:59 smithi017 ceph-mon[31926]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:00.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:01:59 smithi188 ceph-mon[36206]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:00.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:01:59 smithi019 ceph-mon[35944]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:00.956 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:02:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:02:00.630+0000 7effc722b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:02:00.956 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:02:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:02:00.630+0000 7effc722b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:02:00.956 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:02:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:02:00.630+0000 7effc722b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:02:00.956 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:02:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:02:00.630+0000 7effc722b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:02:00.956 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:02:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:02:00.630+0000 7effc722b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-24T03:02:02.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:01 smithi017 ceph-mon[31926]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:02:02.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:01 smithi188 ceph-mon[36206]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:02:02.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:01 smithi019 ceph-mon[35944]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:02:04.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:03 smithi017 ceph-mon[31926]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:02:04.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:03 smithi017 ceph-mon[31926]: mgrmap e24: x(active, since 58s), standbys: y, z 2024-03-24T03:02:04.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:03 smithi188 ceph-mon[36206]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:02:04.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:03 smithi188 ceph-mon[36206]: mgrmap e24: x(active, since 58s), standbys: y, z 2024-03-24T03:02:04.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:03 smithi019 ceph-mon[35944]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:02:04.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:03 smithi019 ceph-mon[35944]: mgrmap e24: x(active, since 58s), standbys: y, z 2024-03-24T03:02:06.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:05 smithi188 ceph-mon[36206]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:06.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:05 smithi019 ceph-mon[35944]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:06.455 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:05 smithi017 ceph-mon[31926]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:08.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:07 smithi188 ceph-mon[36206]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-24T03:02:08.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:07 smithi019 ceph-mon[35944]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-24T03:02:08.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:07 smithi017 ceph-mon[31926]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-24T03:02:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:09 smithi188 ceph-mon[36206]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-24T03:02:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:02:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:09 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:09 smithi019 ceph-mon[35944]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-24T03:02:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:02:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:09 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:09 smithi017 ceph-mon[31926]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-24T03:02:10.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:02:10.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:10.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:09 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:11.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:10 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:02:11.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:10 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:02:11.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:10 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:11.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:10 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:02:11.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:10 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:02:11.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:10 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:11.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:10 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:02:11.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:10 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:02:11.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:10 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:02:12.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:11 smithi188 ceph-mon[36206]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-24T03:02:12.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:11 smithi019 ceph-mon[35944]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-24T03:02:12.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:11 smithi017 ceph-mon[31926]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-24T03:02:14.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:13 smithi188 ceph-mon[36206]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-24T03:02:14.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:13 smithi019 ceph-mon[35944]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-24T03:02:14.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:13 smithi017 ceph-mon[31926]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-24T03:02:16.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:15 smithi188 ceph-mon[36206]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-24T03:02:16.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:15 smithi019 ceph-mon[35944]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-24T03:02:16.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:15 smithi017 ceph-mon[31926]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-24T03:02:18.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:17 smithi188 ceph-mon[36206]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-24T03:02:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:17 smithi019 ceph-mon[35944]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-24T03:02:18.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:17 smithi017 ceph-mon[31926]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-24T03:02:20.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:19 smithi188 ceph-mon[36206]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-24T03:02:20.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:19 smithi019 ceph-mon[35944]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-24T03:02:20.455 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:19 smithi017 ceph-mon[31926]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-24T03:02:22.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:21 smithi188 ceph-mon[36206]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-24T03:02:22.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:21 smithi019 ceph-mon[35944]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-24T03:02:22.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:21 smithi017 ceph-mon[31926]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-24T03:02:24.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:23 smithi188 ceph-mon[36206]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:24.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:23 smithi019 ceph-mon[35944]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:24.455 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:23 smithi017 ceph-mon[31926]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:26.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:25 smithi188 ceph-mon[36206]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:26.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:25 smithi019 ceph-mon[35944]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:26.455 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:25 smithi017 ceph-mon[31926]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:28.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:27 smithi188 ceph-mon[36206]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:02:28.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:27 smithi019 ceph-mon[35944]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:02:28.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:27 smithi017 ceph-mon[31926]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:02:30.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:29 smithi188 ceph-mon[36206]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:29 smithi019 ceph-mon[35944]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:30.455 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:29 smithi017 ceph-mon[31926]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:31.475 INFO:tasks.workunit.client.0.smithi017.stderr:Updating files: 96% (11795/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-24T03:02:32.044 INFO:tasks.workunit.client.0.smithi017.stderr:Note: switching to '3be9a5c9bc793e11e2800a8c0c696e8b46742033'. 2024-03-24T03:02:32.044 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-03-24T03:02:32.044 INFO:tasks.workunit.client.0.smithi017.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-24T03:02:32.044 INFO:tasks.workunit.client.0.smithi017.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-24T03:02:32.044 INFO:tasks.workunit.client.0.smithi017.stderr:state without impacting any branches by switching back to a branch. 2024-03-24T03:02:32.044 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-03-24T03:02:32.044 INFO:tasks.workunit.client.0.smithi017.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-24T03:02:32.044 INFO:tasks.workunit.client.0.smithi017.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-24T03:02:32.044 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-03-24T03:02:32.045 INFO:tasks.workunit.client.0.smithi017.stderr: git switch -c 2024-03-24T03:02:32.045 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-03-24T03:02:32.045 INFO:tasks.workunit.client.0.smithi017.stderr:Or undo this operation with: 2024-03-24T03:02:32.045 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-03-24T03:02:32.045 INFO:tasks.workunit.client.0.smithi017.stderr: git switch - 2024-03-24T03:02:32.045 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-03-24T03:02:32.045 INFO:tasks.workunit.client.0.smithi017.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-24T03:02:32.045 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-03-24T03:02:32.045 INFO:tasks.workunit.client.0.smithi017.stderr:HEAD is now at 3be9a5c9bc7 dummy commit #1 2024-03-24T03:02:32.048 DEBUG:teuthology.orchestra.run.smithi017:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-24T03:02:32.115 INFO:tasks.workunit.client.0.smithi017.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-24T03:02:32.117 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-24T03:02:32.117 INFO:tasks.workunit.client.0.smithi017.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-24T03:02:32.257 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:31 smithi017 ceph-mon[31926]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:02:32.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:31 smithi188 ceph-mon[36206]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:02:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:31 smithi019 ceph-mon[35944]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:02:32.875 INFO:tasks.workunit.client.0.smithi017.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-24T03:02:32.941 INFO:tasks.workunit.client.0.smithi017.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-24T03:02:32.982 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-24T03:02:32.984 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-24T03:02:32.984 INFO:tasks.workunit.client.0.smithi017.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-24T03:02:33.026 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-24T03:02:33.029 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T03:02:33.029 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-24T03:02:33.085 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-03-24T03:02:33.086 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-03-24T03:02:33.086 DEBUG:teuthology.orchestra.run.smithi017:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-03-24T03:02:33.146 INFO:tasks.workunit.client.0.smithi017.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-03-24T03:02:33.299 INFO:tasks.workunit.client.0.smithi017.stdout: Iozone: Performance Test of File I/O 2024-03-24T03:02:33.299 INFO:tasks.workunit.client.0.smithi017.stdout: Version $Revision: 3.494 $ 2024-03-24T03:02:33.299 INFO:tasks.workunit.client.0.smithi017.stdout: Compiled for 64 bit mode. 2024-03-24T03:02:33.299 INFO:tasks.workunit.client.0.smithi017.stdout: Build: linux-AMD64 2024-03-24T03:02:33.299 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: Alexey Skidanov, Sudhir Kumar. 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: Run began: Sun Mar 24 03:02:33 2024 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: Include close in write timing 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: Include fsync in write timing 2024-03-24T03:02:33.300 INFO:tasks.workunit.client.0.smithi017.stdout: File size set to 1048576 kB 2024-03-24T03:02:33.301 INFO:tasks.workunit.client.0.smithi017.stdout: Record Size 16 kB 2024-03-24T03:02:33.301 INFO:tasks.workunit.client.0.smithi017.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-03-24T03:02:33.301 INFO:tasks.workunit.client.0.smithi017.stdout: Output is in kBytes/sec 2024-03-24T03:02:33.301 INFO:tasks.workunit.client.0.smithi017.stdout: Time Resolution = 0.000001 seconds. 2024-03-24T03:02:33.301 INFO:tasks.workunit.client.0.smithi017.stdout: Processor cache size set to 1024 kBytes. 2024-03-24T03:02:33.301 INFO:tasks.workunit.client.0.smithi017.stdout: Processor cache line size set to 32 bytes. 2024-03-24T03:02:33.301 INFO:tasks.workunit.client.0.smithi017.stdout: File stride size set to 17 * record size. 2024-03-24T03:02:33.301 INFO:tasks.workunit.client.0.smithi017.stdout: Throughput test with 1 process 2024-03-24T03:02:33.301 INFO:tasks.workunit.client.0.smithi017.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-03-24T03:02:34.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:34 smithi188 ceph-mon[36206]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:34.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:34 smithi019 ceph-mon[35944]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:34.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:33 smithi017 ceph-mon[31926]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:34.679 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:34.679 INFO:tasks.workunit.client.0.smithi017.stdout: Children see throughput for 1 initial writers = 761167.00 kB/sec 2024-03-24T03:02:34.679 INFO:tasks.workunit.client.0.smithi017.stdout: Parent sees throughput for 1 initial writers = 761062.44 kB/sec 2024-03-24T03:02:34.679 INFO:tasks.workunit.client.0.smithi017.stdout: Min throughput per process = 761167.00 kB/sec 2024-03-24T03:02:34.679 INFO:tasks.workunit.client.0.smithi017.stdout: Max throughput per process = 761167.00 kB/sec 2024-03-24T03:02:34.679 INFO:tasks.workunit.client.0.smithi017.stdout: Avg throughput per process = 761167.00 kB/sec 2024-03-24T03:02:34.680 INFO:tasks.workunit.client.0.smithi017.stdout: Min xfer = 1048576.00 kB 2024-03-24T03:02:36.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:36 smithi188 ceph-mon[36206]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:36 smithi188 ceph-mon[36206]: mgrmap e25: x(active, since 90s), standbys: y, z 2024-03-24T03:02:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:36 smithi019 ceph-mon[35944]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:36 smithi019 ceph-mon[35944]: mgrmap e25: x(active, since 90s), standbys: y, z 2024-03-24T03:02:36.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:36 smithi017 ceph-mon[31926]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:02:36.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:36 smithi017 ceph-mon[31926]: mgrmap e25: x(active, since 90s), standbys: y, z 2024-03-24T03:02:37.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:37 smithi019 ceph-mon[35944]: pgmap v49: 129 pgs: 129 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 36 op/s 2024-03-24T03:02:37.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:37 smithi017 ceph-mon[31926]: pgmap v49: 129 pgs: 129 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 36 op/s 2024-03-24T03:02:37.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:37 smithi188 ceph-mon[36206]: pgmap v49: 129 pgs: 129 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 36 op/s 2024-03-24T03:02:39.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:39 smithi188 ceph-mon[36206]: pgmap v50: 129 pgs: 129 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 85 MiB/s wr, 32 op/s 2024-03-24T03:02:39.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:39 smithi019 ceph-mon[35944]: pgmap v50: 129 pgs: 129 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 85 MiB/s wr, 32 op/s 2024-03-24T03:02:39.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:39 smithi017 ceph-mon[31926]: pgmap v50: 129 pgs: 129 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 85 MiB/s wr, 32 op/s 2024-03-24T03:02:40.098 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:40.098 INFO:tasks.workunit.client.0.smithi017.stdout: Children see throughput for 1 rewriters = 833530.06 kB/sec 2024-03-24T03:02:40.098 INFO:tasks.workunit.client.0.smithi017.stdout: Parent sees throughput for 1 rewriters = 833436.77 kB/sec 2024-03-24T03:02:40.098 INFO:tasks.workunit.client.0.smithi017.stdout: Min throughput per process = 833530.06 kB/sec 2024-03-24T03:02:40.098 INFO:tasks.workunit.client.0.smithi017.stdout: Max throughput per process = 833530.06 kB/sec 2024-03-24T03:02:40.098 INFO:tasks.workunit.client.0.smithi017.stdout: Avg throughput per process = 833530.06 kB/sec 2024-03-24T03:02:40.099 INFO:tasks.workunit.client.0.smithi017.stdout: Min xfer = 1048576.00 kB 2024-03-24T03:02:41.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:41 smithi188 ceph-mon[36206]: pgmap v51: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 58 op/s 2024-03-24T03:02:41.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:41 smithi019 ceph-mon[35944]: pgmap v51: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 58 op/s 2024-03-24T03:02:41.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:41 smithi017 ceph-mon[31926]: pgmap v51: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 58 op/s 2024-03-24T03:02:42.924 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:42.924 INFO:tasks.workunit.client.0.smithi017.stdout: Children see throughput for 1 readers = 7299422.00 kB/sec 2024-03-24T03:02:42.924 INFO:tasks.workunit.client.0.smithi017.stdout: Parent sees throughput for 1 readers = 7289163.38 kB/sec 2024-03-24T03:02:42.924 INFO:tasks.workunit.client.0.smithi017.stdout: Min throughput per process = 7299422.00 kB/sec 2024-03-24T03:02:42.924 INFO:tasks.workunit.client.0.smithi017.stdout: Max throughput per process = 7299422.00 kB/sec 2024-03-24T03:02:42.924 INFO:tasks.workunit.client.0.smithi017.stdout: Avg throughput per process = 7299422.00 kB/sec 2024-03-24T03:02:42.924 INFO:tasks.workunit.client.0.smithi017.stdout: Min xfer = 1048576.00 kB 2024-03-24T03:02:43.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:43 smithi188 ceph-mon[36206]: pgmap v52: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 171 MiB/s wr, 54 op/s 2024-03-24T03:02:43.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:43 smithi019 ceph-mon[35944]: pgmap v52: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 171 MiB/s wr, 54 op/s 2024-03-24T03:02:43.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:43 smithi017 ceph-mon[31926]: pgmap v52: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 171 MiB/s wr, 54 op/s 2024-03-24T03:02:45.134 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:45.135 INFO:tasks.workunit.client.0.smithi017.stdout: Children see throughput for 1 re-readers = 7623733.50 kB/sec 2024-03-24T03:02:45.135 INFO:tasks.workunit.client.0.smithi017.stdout: Parent sees throughput for 1 re-readers = 7613610.85 kB/sec 2024-03-24T03:02:45.135 INFO:tasks.workunit.client.0.smithi017.stdout: Min throughput per process = 7623733.50 kB/sec 2024-03-24T03:02:45.135 INFO:tasks.workunit.client.0.smithi017.stdout: Max throughput per process = 7623733.50 kB/sec 2024-03-24T03:02:45.135 INFO:tasks.workunit.client.0.smithi017.stdout: Avg throughput per process = 7623733.50 kB/sec 2024-03-24T03:02:45.135 INFO:tasks.workunit.client.0.smithi017.stdout: Min xfer = 1048576.00 kB 2024-03-24T03:02:45.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:45 smithi188 ceph-mon[36206]: pgmap v53: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 171 MiB/s wr, 54 op/s 2024-03-24T03:02:45.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:45 smithi019 ceph-mon[35944]: pgmap v53: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 171 MiB/s wr, 54 op/s 2024-03-24T03:02:45.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:45 smithi017 ceph-mon[31926]: pgmap v53: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 171 MiB/s wr, 54 op/s 2024-03-24T03:02:47.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:47 smithi188 ceph-mon[36206]: pgmap v54: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 58 op/s 2024-03-24T03:02:47.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:47 smithi019 ceph-mon[35944]: pgmap v54: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 58 op/s 2024-03-24T03:02:47.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:47 smithi017 ceph-mon[31926]: pgmap v54: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 58 op/s 2024-03-24T03:02:49.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:49 smithi188 ceph-mon[36206]: pgmap v55: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 MiB/s wr, 30 op/s 2024-03-24T03:02:49.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:49 smithi019 ceph-mon[35944]: pgmap v55: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 MiB/s wr, 30 op/s 2024-03-24T03:02:49.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:49 smithi017 ceph-mon[31926]: pgmap v55: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 MiB/s wr, 30 op/s 2024-03-24T03:02:51.507 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:51.507 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:51.507 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:51.507 INFO:tasks.workunit.client.0.smithi017.stdout:iozone test complete. 2024-03-24T03:02:51.507 INFO:tasks.workunit.client.0.smithi017.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-03-24T03:02:51.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:51 smithi188 ceph-mon[36206]: pgmap v56: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-03-24T03:02:51.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:51 smithi019 ceph-mon[35944]: pgmap v56: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-03-24T03:02:51.596 INFO:tasks.workunit.client.0.smithi017.stdout: Iozone: Performance Test of File I/O 2024-03-24T03:02:51.596 INFO:tasks.workunit.client.0.smithi017.stdout: Version $Revision: 3.494 $ 2024-03-24T03:02:51.596 INFO:tasks.workunit.client.0.smithi017.stdout: Compiled for 64 bit mode. 2024-03-24T03:02:51.597 INFO:tasks.workunit.client.0.smithi017.stdout: Build: linux-AMD64 2024-03-24T03:02:51.597 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:51.597 INFO:tasks.workunit.client.0.smithi017.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-03-24T03:02:51.597 INFO:tasks.workunit.client.0.smithi017.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-03-24T03:02:51.597 INFO:tasks.workunit.client.0.smithi017.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-03-24T03:02:51.597 INFO:tasks.workunit.client.0.smithi017.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-03-24T03:02:51.597 INFO:tasks.workunit.client.0.smithi017.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-03-24T03:02:51.597 INFO:tasks.workunit.client.0.smithi017.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-03-24T03:02:51.597 INFO:tasks.workunit.client.0.smithi017.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-03-24T03:02:51.597 INFO:tasks.workunit.client.0.smithi017.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-03-24T03:02:51.598 INFO:tasks.workunit.client.0.smithi017.stdout: Alexey Skidanov, Sudhir Kumar. 2024-03-24T03:02:51.598 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:51.598 INFO:tasks.workunit.client.0.smithi017.stdout: Run began: Sun Mar 24 03:02:51 2024 2024-03-24T03:02:51.598 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:51.598 INFO:tasks.workunit.client.0.smithi017.stdout: Include close in write timing 2024-03-24T03:02:51.598 INFO:tasks.workunit.client.0.smithi017.stdout: Include fsync in write timing 2024-03-24T03:02:51.598 INFO:tasks.workunit.client.0.smithi017.stdout: File size set to 1048576 kB 2024-03-24T03:02:51.598 INFO:tasks.workunit.client.0.smithi017.stdout: Record Size 1024 kB 2024-03-24T03:02:51.598 INFO:tasks.workunit.client.0.smithi017.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-03-24T03:02:51.598 INFO:tasks.workunit.client.0.smithi017.stdout: Output is in kBytes/sec 2024-03-24T03:02:51.598 INFO:tasks.workunit.client.0.smithi017.stdout: Time Resolution = 0.000001 seconds. 2024-03-24T03:02:51.599 INFO:tasks.workunit.client.0.smithi017.stdout: Processor cache size set to 1024 kBytes. 2024-03-24T03:02:51.599 INFO:tasks.workunit.client.0.smithi017.stdout: Processor cache line size set to 32 bytes. 2024-03-24T03:02:51.599 INFO:tasks.workunit.client.0.smithi017.stdout: File stride size set to 17 * record size. 2024-03-24T03:02:51.599 INFO:tasks.workunit.client.0.smithi017.stdout: Throughput test with 1 process 2024-03-24T03:02:51.599 INFO:tasks.workunit.client.0.smithi017.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-03-24T03:02:51.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:51 smithi017 ceph-mon[31926]: pgmap v56: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-03-24T03:02:52.967 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:52.968 INFO:tasks.workunit.client.0.smithi017.stdout: Children see throughput for 1 initial writers = 785531.38 kB/sec 2024-03-24T03:02:52.968 INFO:tasks.workunit.client.0.smithi017.stdout: Parent sees throughput for 1 initial writers = 785405.59 kB/sec 2024-03-24T03:02:52.968 INFO:tasks.workunit.client.0.smithi017.stdout: Min throughput per process = 785531.38 kB/sec 2024-03-24T03:02:52.968 INFO:tasks.workunit.client.0.smithi017.stdout: Max throughput per process = 785531.38 kB/sec 2024-03-24T03:02:52.968 INFO:tasks.workunit.client.0.smithi017.stdout: Avg throughput per process = 785531.38 kB/sec 2024-03-24T03:02:52.968 INFO:tasks.workunit.client.0.smithi017.stdout: Min xfer = 1048576.00 kB 2024-03-24T03:02:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:53 smithi188 ceph-mon[36206]: pgmap v57: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-24T03:02:53.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:53 smithi019 ceph-mon[35944]: pgmap v57: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-24T03:02:53.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:53 smithi017 ceph-mon[31926]: pgmap v57: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-24T03:02:55.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:55 smithi188 ceph-mon[36206]: pgmap v58: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-24T03:02:55.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:55 smithi019 ceph-mon[35944]: pgmap v58: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-24T03:02:55.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:55 smithi017 ceph-mon[31926]: pgmap v58: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-24T03:02:56.820 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:56.820 INFO:tasks.workunit.client.0.smithi017.stdout: Children see throughput for 1 rewriters = 830647.31 kB/sec 2024-03-24T03:02:56.820 INFO:tasks.workunit.client.0.smithi017.stdout: Parent sees throughput for 1 rewriters = 830475.11 kB/sec 2024-03-24T03:02:56.821 INFO:tasks.workunit.client.0.smithi017.stdout: Min throughput per process = 830647.31 kB/sec 2024-03-24T03:02:56.821 INFO:tasks.workunit.client.0.smithi017.stdout: Max throughput per process = 830647.31 kB/sec 2024-03-24T03:02:56.821 INFO:tasks.workunit.client.0.smithi017.stdout: Avg throughput per process = 830647.31 kB/sec 2024-03-24T03:02:56.821 INFO:tasks.workunit.client.0.smithi017.stdout: Min xfer = 1048576.00 kB 2024-03-24T03:02:57.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:57 smithi188 ceph-mon[36206]: pgmap v59: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 85 MiB/s wr, 57 op/s 2024-03-24T03:02:57.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:57 smithi019 ceph-mon[35944]: pgmap v59: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 85 MiB/s wr, 57 op/s 2024-03-24T03:02:57.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:57 smithi017 ceph-mon[31926]: pgmap v59: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 85 MiB/s wr, 57 op/s 2024-03-24T03:02:59.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:02:59 smithi188 ceph-mon[36206]: pgmap v60: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-03-24T03:02:59.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:02:59 smithi019 ceph-mon[35944]: pgmap v60: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-03-24T03:02:59.692 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:02:59.692 INFO:tasks.workunit.client.0.smithi017.stdout: Children see throughput for 1 readers = 7745892.50 kB/sec 2024-03-24T03:02:59.692 INFO:tasks.workunit.client.0.smithi017.stdout: Parent sees throughput for 1 readers = 7734164.33 kB/sec 2024-03-24T03:02:59.692 INFO:tasks.workunit.client.0.smithi017.stdout: Min throughput per process = 7745892.50 kB/sec 2024-03-24T03:02:59.692 INFO:tasks.workunit.client.0.smithi017.stdout: Max throughput per process = 7745892.50 kB/sec 2024-03-24T03:02:59.693 INFO:tasks.workunit.client.0.smithi017.stdout: Avg throughput per process = 7745892.50 kB/sec 2024-03-24T03:02:59.693 INFO:tasks.workunit.client.0.smithi017.stdout: Min xfer = 1048576.00 kB 2024-03-24T03:02:59.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:02:59 smithi017 ceph-mon[31926]: pgmap v60: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-03-24T03:03:01.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:01 smithi188 ceph-mon[36206]: pgmap v61: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-03-24T03:03:01.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:01 smithi019 ceph-mon[35944]: pgmap v61: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-03-24T03:03:01.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:01 smithi017 ceph-mon[31926]: pgmap v61: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-03-24T03:03:01.907 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:01.907 INFO:tasks.workunit.client.0.smithi017.stdout: Children see throughput for 1 re-readers = 7722783.50 kB/sec 2024-03-24T03:03:01.907 INFO:tasks.workunit.client.0.smithi017.stdout: Parent sees throughput for 1 re-readers = 7708692.44 kB/sec 2024-03-24T03:03:01.907 INFO:tasks.workunit.client.0.smithi017.stdout: Min throughput per process = 7722783.50 kB/sec 2024-03-24T03:03:01.908 INFO:tasks.workunit.client.0.smithi017.stdout: Max throughput per process = 7722783.50 kB/sec 2024-03-24T03:03:01.908 INFO:tasks.workunit.client.0.smithi017.stdout: Avg throughput per process = 7722783.50 kB/sec 2024-03-24T03:03:01.908 INFO:tasks.workunit.client.0.smithi017.stdout: Min xfer = 1048576.00 kB 2024-03-24T03:03:03.508 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:03 smithi017 ceph-mon[31926]: pgmap v62: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 171 MiB/s wr, 74 op/s 2024-03-24T03:03:03.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:03 smithi188 ceph-mon[36206]: pgmap v62: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 171 MiB/s wr, 74 op/s 2024-03-24T03:03:03.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:03 smithi019 ceph-mon[35944]: pgmap v62: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 171 MiB/s wr, 74 op/s 2024-03-24T03:03:05.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:05 smithi188 ceph-mon[36206]: pgmap v63: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 171 MiB/s wr, 74 op/s 2024-03-24T03:03:05.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:05 smithi019 ceph-mon[35944]: pgmap v63: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 171 MiB/s wr, 74 op/s 2024-03-24T03:03:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:05 smithi017 ceph-mon[31926]: pgmap v63: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 171 MiB/s wr, 74 op/s 2024-03-24T03:03:07.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:07 smithi188 ceph-mon[36206]: pgmap v64: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 171 MiB/s wr, 94 op/s 2024-03-24T03:03:07.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:07 smithi019 ceph-mon[35944]: pgmap v64: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 171 MiB/s wr, 94 op/s 2024-03-24T03:03:07.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:07 smithi017 ceph-mon[31926]: pgmap v64: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 171 MiB/s wr, 94 op/s 2024-03-24T03:03:08.838 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:08.838 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:08.838 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:08.838 INFO:tasks.workunit.client.0.smithi017.stdout:iozone test complete. 2024-03-24T03:03:08.838 INFO:tasks.workunit.client.0.smithi017.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-03-24T03:03:08.905 INFO:tasks.workunit.client.0.smithi017.stdout: Iozone: Performance Test of File I/O 2024-03-24T03:03:08.905 INFO:tasks.workunit.client.0.smithi017.stdout: Version $Revision: 3.494 $ 2024-03-24T03:03:08.905 INFO:tasks.workunit.client.0.smithi017.stdout: Compiled for 64 bit mode. 2024-03-24T03:03:08.905 INFO:tasks.workunit.client.0.smithi017.stdout: Build: linux-AMD64 2024-03-24T03:03:08.905 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:08.905 INFO:tasks.workunit.client.0.smithi017.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-03-24T03:03:08.905 INFO:tasks.workunit.client.0.smithi017.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: Alexey Skidanov, Sudhir Kumar. 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: Run began: Sun Mar 24 03:03:08 2024 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: Include close in write timing 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: Include fsync in write timing 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: File size set to 10485760 kB 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: Record Size 1024 kB 2024-03-24T03:03:08.906 INFO:tasks.workunit.client.0.smithi017.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-03-24T03:03:08.907 INFO:tasks.workunit.client.0.smithi017.stdout: Output is in kBytes/sec 2024-03-24T03:03:08.907 INFO:tasks.workunit.client.0.smithi017.stdout: Time Resolution = 0.000001 seconds. 2024-03-24T03:03:08.907 INFO:tasks.workunit.client.0.smithi017.stdout: Processor cache size set to 1024 kBytes. 2024-03-24T03:03:08.907 INFO:tasks.workunit.client.0.smithi017.stdout: Processor cache line size set to 32 bytes. 2024-03-24T03:03:08.907 INFO:tasks.workunit.client.0.smithi017.stdout: File stride size set to 17 * record size. 2024-03-24T03:03:08.907 INFO:tasks.workunit.client.0.smithi017.stdout: Throughput test with 1 process 2024-03-24T03:03:08.907 INFO:tasks.workunit.client.0.smithi017.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-03-24T03:03:09.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:09 smithi188 ceph-mon[36206]: pgmap v65: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 85 MiB/s wr, 45 op/s 2024-03-24T03:03:09.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:09 smithi019 ceph-mon[35944]: pgmap v65: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 85 MiB/s wr, 45 op/s 2024-03-24T03:03:09.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:09 smithi017 ceph-mon[31926]: pgmap v65: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 85 MiB/s wr, 45 op/s 2024-03-24T03:03:11.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:11 smithi188 ceph-mon[36206]: pgmap v66: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 115 MiB/s wr, 96 op/s 2024-03-24T03:03:11.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:11 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:03:11.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:11 smithi019 ceph-mon[35944]: pgmap v66: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 115 MiB/s wr, 96 op/s 2024-03-24T03:03:11.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:11 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:03:11.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:11 smithi017 ceph-mon[31926]: pgmap v66: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 115 MiB/s wr, 96 op/s 2024-03-24T03:03:11.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:11 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:03:12.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:12 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:03:12.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:12 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:03:12.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:12 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:03:12.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:12 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:03:12.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:12 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:03:12.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:12 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:03:12.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:12 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:03:12.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:12 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:03:12.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:12 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:03:13.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:13 smithi188 ceph-mon[36206]: pgmap v67: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 30 MiB/s wr, 70 op/s 2024-03-24T03:03:13.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:13 smithi019 ceph-mon[35944]: pgmap v67: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 30 MiB/s wr, 70 op/s 2024-03-24T03:03:13.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:13 smithi017 ceph-mon[31926]: pgmap v67: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 30 MiB/s wr, 70 op/s 2024-03-24T03:03:15.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:15 smithi019 ceph-mon[35944]: pgmap v68: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 30 MiB/s wr, 70 op/s 2024-03-24T03:03:15.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:15 smithi017 ceph-mon[31926]: pgmap v68: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 30 MiB/s wr, 70 op/s 2024-03-24T03:03:15.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:15 smithi188 ceph-mon[36206]: pgmap v68: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 30 MiB/s wr, 70 op/s 2024-03-24T03:03:17.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:17 smithi019 ceph-mon[35944]: pgmap v69: 129 pgs: 129 active+clean; 5.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 405 MiB/s wr, 341 op/s 2024-03-24T03:03:17.600 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:17 smithi188 ceph-mon[36206]: pgmap v69: 129 pgs: 129 active+clean; 5.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 405 MiB/s wr, 341 op/s 2024-03-24T03:03:17.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:17 smithi017 ceph-mon[31926]: pgmap v69: 129 pgs: 129 active+clean; 5.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 405 MiB/s wr, 341 op/s 2024-03-24T03:03:19.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:19 smithi017 ceph-mon[31926]: pgmap v70: 129 pgs: 129 active+clean; 5.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 405 MiB/s wr, 322 op/s 2024-03-24T03:03:19.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:19 smithi188 ceph-mon[36206]: pgmap v70: 129 pgs: 129 active+clean; 5.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 405 MiB/s wr, 322 op/s 2024-03-24T03:03:19.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:19 smithi019 ceph-mon[35944]: pgmap v70: 129 pgs: 129 active+clean; 5.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 405 MiB/s wr, 322 op/s 2024-03-24T03:03:20.478 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:20.478 INFO:tasks.workunit.client.0.smithi017.stdout: Children see throughput for 1 initial writers = 908842.94 kB/sec 2024-03-24T03:03:20.478 INFO:tasks.workunit.client.0.smithi017.stdout: Parent sees throughput for 1 initial writers = 908822.53 kB/sec 2024-03-24T03:03:20.478 INFO:tasks.workunit.client.0.smithi017.stdout: Min throughput per process = 908842.94 kB/sec 2024-03-24T03:03:20.478 INFO:tasks.workunit.client.0.smithi017.stdout: Max throughput per process = 908842.94 kB/sec 2024-03-24T03:03:20.478 INFO:tasks.workunit.client.0.smithi017.stdout: Avg throughput per process = 908842.94 kB/sec 2024-03-24T03:03:20.479 INFO:tasks.workunit.client.0.smithi017.stdout: Min xfer = 10485760.00 kB 2024-03-24T03:03:21.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:21 smithi017 ceph-mon[31926]: pgmap v71: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 751 MiB/s wr, 444 op/s 2024-03-24T03:03:21.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:21 smithi188 ceph-mon[36206]: pgmap v71: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 751 MiB/s wr, 444 op/s 2024-03-24T03:03:21.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:21 smithi019 ceph-mon[35944]: pgmap v71: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 751 MiB/s wr, 444 op/s 2024-03-24T03:03:23.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:23 smithi017 ceph-mon[31926]: pgmap v72: 129 pgs: 129 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 803 MiB/s wr, 413 op/s 2024-03-24T03:03:23.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:23 smithi188 ceph-mon[36206]: pgmap v72: 129 pgs: 129 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 803 MiB/s wr, 413 op/s 2024-03-24T03:03:23.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:23 smithi019 ceph-mon[35944]: pgmap v72: 129 pgs: 129 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 803 MiB/s wr, 413 op/s 2024-03-24T03:03:25.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:25 smithi017 ceph-mon[31926]: pgmap v73: 129 pgs: 129 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 803 MiB/s wr, 413 op/s 2024-03-24T03:03:25.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:25 smithi188 ceph-mon[36206]: pgmap v73: 129 pgs: 129 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 803 MiB/s wr, 413 op/s 2024-03-24T03:03:25.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:25 smithi019 ceph-mon[35944]: pgmap v73: 129 pgs: 129 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 803 MiB/s wr, 413 op/s 2024-03-24T03:03:27.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:27 smithi017 ceph-mon[31926]: pgmap v74: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 895 MiB/s wr, 455 op/s 2024-03-24T03:03:27.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:27 smithi188 ceph-mon[36206]: pgmap v74: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 895 MiB/s wr, 455 op/s 2024-03-24T03:03:27.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:27 smithi019 ceph-mon[35944]: pgmap v74: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 895 MiB/s wr, 455 op/s 2024-03-24T03:03:29.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:29 smithi017 ceph-mon[31926]: pgmap v75: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 520 MiB/s wr, 184 op/s 2024-03-24T03:03:29.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:29 smithi188 ceph-mon[36206]: pgmap v75: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 520 MiB/s wr, 184 op/s 2024-03-24T03:03:29.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:29 smithi019 ceph-mon[35944]: pgmap v75: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 520 MiB/s wr, 184 op/s 2024-03-24T03:03:31.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:31 smithi017 ceph-mon[31926]: pgmap v76: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 708 MiB/s wr, 233 op/s 2024-03-24T03:03:31.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:31 smithi188 ceph-mon[36206]: pgmap v76: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 708 MiB/s wr, 233 op/s 2024-03-24T03:03:31.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:31 smithi019 ceph-mon[35944]: pgmap v76: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 708 MiB/s wr, 233 op/s 2024-03-24T03:03:33.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:33 smithi017 ceph-mon[31926]: pgmap v77: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 556 MiB/s wr, 270 op/s 2024-03-24T03:03:33.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:33 smithi188 ceph-mon[36206]: pgmap v77: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 556 MiB/s wr, 270 op/s 2024-03-24T03:03:33.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:33 smithi019 ceph-mon[35944]: pgmap v77: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 556 MiB/s wr, 270 op/s 2024-03-24T03:03:34.948 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:34.948 INFO:tasks.workunit.client.0.smithi017.stdout: Children see throughput for 1 rewriters = 931280.06 kB/sec 2024-03-24T03:03:34.948 INFO:tasks.workunit.client.0.smithi017.stdout: Parent sees throughput for 1 rewriters = 931260.79 kB/sec 2024-03-24T03:03:34.948 INFO:tasks.workunit.client.0.smithi017.stdout: Min throughput per process = 931280.06 kB/sec 2024-03-24T03:03:34.949 INFO:tasks.workunit.client.0.smithi017.stdout: Max throughput per process = 931280.06 kB/sec 2024-03-24T03:03:34.949 INFO:tasks.workunit.client.0.smithi017.stdout: Avg throughput per process = 931280.06 kB/sec 2024-03-24T03:03:34.949 INFO:tasks.workunit.client.0.smithi017.stdout: Min xfer = 10485760.00 kB 2024-03-24T03:03:35.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:35 smithi017 ceph-mon[31926]: pgmap v78: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 475 MiB/s wr, 249 op/s 2024-03-24T03:03:35.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:35 smithi188 ceph-mon[36206]: pgmap v78: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 475 MiB/s wr, 249 op/s 2024-03-24T03:03:35.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:35 smithi019 ceph-mon[35944]: pgmap v78: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 475 MiB/s wr, 249 op/s 2024-03-24T03:03:37.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:37 smithi017 ceph-mon[31926]: pgmap v79: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 874 MiB/s wr, 364 op/s 2024-03-24T03:03:37.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:37 smithi188 ceph-mon[36206]: pgmap v79: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 874 MiB/s wr, 364 op/s 2024-03-24T03:03:37.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:37 smithi019 ceph-mon[35944]: pgmap v79: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 874 MiB/s wr, 364 op/s 2024-03-24T03:03:39.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:39 smithi017 ceph-mon[31926]: pgmap v80: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 782 MiB/s wr, 323 op/s 2024-03-24T03:03:39.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:39 smithi188 ceph-mon[36206]: pgmap v80: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 782 MiB/s wr, 323 op/s 2024-03-24T03:03:39.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:39 smithi019 ceph-mon[35944]: pgmap v80: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 782 MiB/s wr, 323 op/s 2024-03-24T03:03:41.097 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:41.097 INFO:tasks.workunit.client.0.smithi017.stdout: Children see throughput for 1 readers = 9062769.00 kB/sec 2024-03-24T03:03:41.097 INFO:tasks.workunit.client.0.smithi017.stdout: Parent sees throughput for 1 readers = 9060819.72 kB/sec 2024-03-24T03:03:41.097 INFO:tasks.workunit.client.0.smithi017.stdout: Min throughput per process = 9062769.00 kB/sec 2024-03-24T03:03:41.097 INFO:tasks.workunit.client.0.smithi017.stdout: Max throughput per process = 9062769.00 kB/sec 2024-03-24T03:03:41.097 INFO:tasks.workunit.client.0.smithi017.stdout: Avg throughput per process = 9062769.00 kB/sec 2024-03-24T03:03:41.097 INFO:tasks.workunit.client.0.smithi017.stdout: Min xfer = 10485760.00 kB 2024-03-24T03:03:41.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:41 smithi017 ceph-mon[31926]: pgmap v81: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 782 MiB/s wr, 326 op/s 2024-03-24T03:03:41.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:41 smithi188 ceph-mon[36206]: pgmap v81: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 782 MiB/s wr, 326 op/s 2024-03-24T03:03:41.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:41 smithi019 ceph-mon[35944]: pgmap v81: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 782 MiB/s wr, 326 op/s 2024-03-24T03:03:43.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:43 smithi017 ceph-mon[31926]: pgmap v82: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 594 MiB/s wr, 278 op/s 2024-03-24T03:03:43.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:43 smithi188 ceph-mon[36206]: pgmap v82: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 594 MiB/s wr, 278 op/s 2024-03-24T03:03:43.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:43 smithi019 ceph-mon[35944]: pgmap v82: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 594 MiB/s wr, 278 op/s 2024-03-24T03:03:44.459 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:44.459 INFO:tasks.workunit.client.0.smithi017.stdout: Children see throughput for 1 re-readers = 9103890.00 kB/sec 2024-03-24T03:03:44.459 INFO:tasks.workunit.client.0.smithi017.stdout: Parent sees throughput for 1 re-readers = 9102158.93 kB/sec 2024-03-24T03:03:44.459 INFO:tasks.workunit.client.0.smithi017.stdout: Min throughput per process = 9103890.00 kB/sec 2024-03-24T03:03:44.459 INFO:tasks.workunit.client.0.smithi017.stdout: Max throughput per process = 9103890.00 kB/sec 2024-03-24T03:03:44.459 INFO:tasks.workunit.client.0.smithi017.stdout: Avg throughput per process = 9103890.00 kB/sec 2024-03-24T03:03:44.459 INFO:tasks.workunit.client.0.smithi017.stdout: Min xfer = 10485760.00 kB 2024-03-24T03:03:45.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:45 smithi017 ceph-mon[31926]: pgmap v83: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 400 MiB/s wr, 119 op/s 2024-03-24T03:03:45.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:45 smithi188 ceph-mon[36206]: pgmap v83: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 400 MiB/s wr, 119 op/s 2024-03-24T03:03:45.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:45 smithi019 ceph-mon[35944]: pgmap v83: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 400 MiB/s wr, 119 op/s 2024-03-24T03:03:47.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:47 smithi017 ceph-mon[31926]: pgmap v84: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 400 MiB/s wr, 123 op/s 2024-03-24T03:03:47.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:47 smithi188 ceph-mon[36206]: pgmap v84: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 400 MiB/s wr, 123 op/s 2024-03-24T03:03:47.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:47 smithi019 ceph-mon[35944]: pgmap v84: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 400 MiB/s wr, 123 op/s 2024-03-24T03:03:49.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:49 smithi017 ceph-mon[31926]: pgmap v85: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-24T03:03:49.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:49 smithi188 ceph-mon[36206]: pgmap v85: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-24T03:03:49.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:49 smithi019 ceph-mon[35944]: pgmap v85: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-24T03:03:51.424 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:51.424 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:51.424 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-03-24T03:03:51.424 INFO:tasks.workunit.client.0.smithi017.stdout:iozone test complete. 2024-03-24T03:03:51.425 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-24T03:03:51.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-24T03:03:51.506 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-03-24T03:03:51.506 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-24T03:03:51.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:51 smithi017 ceph-mon[31926]: pgmap v86: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-24T03:03:51.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:51 smithi188 ceph-mon[36206]: pgmap v86: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-24T03:03:51.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:51 smithi019 ceph-mon[35944]: pgmap v86: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-24T03:03:51.829 DEBUG:teuthology.parallel:result is None 2024-03-24T03:03:51.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-24T03:03:51.866 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-24T03:03:51.866 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-24T03:03:51.878 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-24T03:03:51.879 INFO:teuthology.task.exec:Executing custom commands... 2024-03-24T03:03:51.879 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2024-03-24T03:03:51.879 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-24T03:03:51.934 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-24T03:03:53.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:53 smithi017 ceph-mon[31926]: pgmap v87: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-24T03:03:53.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:53 smithi188 ceph-mon[36206]: pgmap v87: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-24T03:03:53.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:53 smithi019 ceph-mon[35944]: pgmap v87: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-24T03:03:55.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:55 smithi017 ceph-mon[31926]: pgmap v88: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-24T03:03:55.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:55 smithi188 ceph-mon[36206]: pgmap v88: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-24T03:03:55.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:55 smithi019 ceph-mon[35944]: pgmap v88: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-24T03:03:57.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:57 smithi017 ceph-mon[31926]: pgmap v89: 129 pgs: 129 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 3.7 KiB/s wr, 227 op/s 2024-03-24T03:03:57.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:57 smithi188 ceph-mon[36206]: pgmap v89: 129 pgs: 129 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 3.7 KiB/s wr, 227 op/s 2024-03-24T03:03:57.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:57 smithi019 ceph-mon[35944]: pgmap v89: 129 pgs: 129 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 3.7 KiB/s wr, 227 op/s 2024-03-24T03:03:59.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:03:59 smithi017 ceph-mon[31926]: pgmap v90: 129 pgs: 129 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 223 op/s 2024-03-24T03:03:59.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:03:59 smithi188 ceph-mon[36206]: pgmap v90: 129 pgs: 129 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 223 op/s 2024-03-24T03:03:59.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:03:59 smithi019 ceph-mon[35944]: pgmap v90: 129 pgs: 129 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 223 op/s 2024-03-24T03:04:01.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:01 smithi017 ceph-mon[31926]: pgmap v91: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 226 op/s 2024-03-24T03:04:01.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:01 smithi188 ceph-mon[36206]: pgmap v91: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 226 op/s 2024-03-24T03:04:01.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:01 smithi019 ceph-mon[35944]: pgmap v91: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 226 op/s 2024-03-24T03:04:03.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:03 smithi017 ceph-mon[31926]: pgmap v92: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.1 KiB/s wr, 224 op/s 2024-03-24T03:04:03.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:03 smithi188 ceph-mon[36206]: pgmap v92: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.1 KiB/s wr, 224 op/s 2024-03-24T03:04:03.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:03 smithi019 ceph-mon[35944]: pgmap v92: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.1 KiB/s wr, 224 op/s 2024-03-24T03:04:05.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:05 smithi017 ceph-mon[31926]: pgmap v93: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.1 KiB/s wr, 222 op/s 2024-03-24T03:04:05.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:05 smithi188 ceph-mon[36206]: pgmap v93: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.1 KiB/s wr, 222 op/s 2024-03-24T03:04:05.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:05 smithi019 ceph-mon[35944]: pgmap v93: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.1 KiB/s wr, 222 op/s 2024-03-24T03:04:07.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:07 smithi017 ceph-mon[31926]: pgmap v94: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 KiB/s wr, 241 op/s 2024-03-24T03:04:07.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:07 smithi188 ceph-mon[36206]: pgmap v94: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 KiB/s wr, 241 op/s 2024-03-24T03:04:07.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:07 smithi019 ceph-mon[35944]: pgmap v94: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 KiB/s wr, 241 op/s 2024-03-24T03:04:09.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:09 smithi017 ceph-mon[31926]: pgmap v95: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-24T03:04:09.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:09 smithi188 ceph-mon[36206]: pgmap v95: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-24T03:04:09.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:09 smithi019 ceph-mon[35944]: pgmap v95: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-24T03:04:11.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:11 smithi017 ceph-mon[31926]: pgmap v96: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-03-24T03:04:11.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:11 smithi188 ceph-mon[36206]: pgmap v96: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-03-24T03:04:11.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:11 smithi019 ceph-mon[35944]: pgmap v96: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-03-24T03:04:12.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:12 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:04:12.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:12 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:04:12.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:12 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:04:13.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:13 smithi017 ceph-mon[31926]: pgmap v97: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-24T03:04:13.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:13 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:04:13.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:13 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:04:13.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:13 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:04:13.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:13 smithi188 ceph-mon[36206]: pgmap v97: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-24T03:04:13.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:13 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:04:13.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:13 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:04:13.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:13 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:04:13.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:13 smithi019 ceph-mon[35944]: pgmap v97: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-24T03:04:13.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:13 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:04:13.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:13 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:04:13.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:13 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:04:15.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:15 smithi017 ceph-mon[31926]: pgmap v98: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:04:15.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:15 smithi188 ceph-mon[36206]: pgmap v98: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:04:15.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:15 smithi019 ceph-mon[35944]: pgmap v98: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:04:17.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:17 smithi188 ceph-mon[36206]: pgmap v99: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:04:17.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:17 smithi019 ceph-mon[35944]: pgmap v99: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:04:17.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:17 smithi017 ceph-mon[31926]: pgmap v99: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:04:19.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:19 smithi188 ceph-mon[36206]: pgmap v100: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-24T03:04:19.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:19 smithi019 ceph-mon[35944]: pgmap v100: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-24T03:04:19.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:19 smithi017 ceph-mon[31926]: pgmap v100: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-24T03:04:21.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:21 smithi188 ceph-mon[36206]: pgmap v101: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-24T03:04:21.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:21 smithi019 ceph-mon[35944]: pgmap v101: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-24T03:04:21.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:21 smithi017 ceph-mon[31926]: pgmap v101: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-24T03:04:23.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:23 smithi188 ceph-mon[36206]: pgmap v102: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-24T03:04:23.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:23 smithi019 ceph-mon[35944]: pgmap v102: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-24T03:04:23.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:23 smithi017 ceph-mon[31926]: pgmap v102: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-24T03:04:25.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:25 smithi188 ceph-mon[36206]: pgmap v103: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:04:25.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:25 smithi019 ceph-mon[35944]: pgmap v103: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:04:25.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:25 smithi017 ceph-mon[31926]: pgmap v103: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:04:27.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:27 smithi188 ceph-mon[36206]: pgmap v104: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:04:27.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:27 smithi019 ceph-mon[35944]: pgmap v104: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:04:27.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:27 smithi017 ceph-mon[31926]: pgmap v104: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:04:29.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:29 smithi188 ceph-mon[36206]: pgmap v105: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:04:29.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:29 smithi019 ceph-mon[35944]: pgmap v105: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:04:29.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:29 smithi017 ceph-mon[31926]: pgmap v105: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:04:31.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:31 smithi188 ceph-mon[36206]: pgmap v106: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:04:31.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:31 smithi019 ceph-mon[35944]: pgmap v106: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:04:31.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:31 smithi017 ceph-mon[31926]: pgmap v106: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:04:33.758 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:33 smithi017 ceph-mon[31926]: pgmap v107: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:04:33.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:33 smithi188 ceph-mon[36206]: pgmap v107: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:04:33.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:33 smithi019 ceph-mon[35944]: pgmap v107: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:04:35.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:35 smithi188 ceph-mon[36206]: pgmap v108: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:35.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:35 smithi019 ceph-mon[35944]: pgmap v108: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:35.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:35 smithi017 ceph-mon[31926]: pgmap v108: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:37.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:37 smithi188 ceph-mon[36206]: pgmap v109: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:04:37.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:37 smithi019 ceph-mon[35944]: pgmap v109: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:04:37.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:37 smithi017 ceph-mon[31926]: pgmap v109: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:04:39.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:39 smithi188 ceph-mon[36206]: pgmap v110: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:39.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:39 smithi019 ceph-mon[35944]: pgmap v110: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:39.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:39 smithi017 ceph-mon[31926]: pgmap v110: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:41.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:41 smithi188 ceph-mon[36206]: pgmap v111: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:04:41.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:41 smithi019 ceph-mon[35944]: pgmap v111: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:04:41.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:41 smithi017 ceph-mon[31926]: pgmap v111: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:04:43.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:43 smithi188 ceph-mon[36206]: pgmap v112: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:04:43.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:43 smithi019 ceph-mon[35944]: pgmap v112: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:04:43.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:43 smithi017 ceph-mon[31926]: pgmap v112: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:04:45.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:45 smithi188 ceph-mon[36206]: pgmap v113: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:45.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:45 smithi019 ceph-mon[35944]: pgmap v113: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:45.955 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:45 smithi017 ceph-mon[31926]: pgmap v113: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:47.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:47 smithi188 ceph-mon[36206]: pgmap v114: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:04:47.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:47 smithi019 ceph-mon[35944]: pgmap v114: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:04:47.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:47 smithi017 ceph-mon[31926]: pgmap v114: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:04:49.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:49 smithi188 ceph-mon[36206]: pgmap v115: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:49.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:49 smithi019 ceph-mon[35944]: pgmap v115: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:49.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:49 smithi017 ceph-mon[31926]: pgmap v115: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:51.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:51 smithi188 ceph-mon[36206]: pgmap v116: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:04:51.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:51 smithi019 ceph-mon[35944]: pgmap v116: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:04:51.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:51 smithi017 ceph-mon[31926]: pgmap v116: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:04:53.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:53 smithi188 ceph-mon[36206]: pgmap v117: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:04:53.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:53 smithi019 ceph-mon[35944]: pgmap v117: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:04:53.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:53 smithi017 ceph-mon[31926]: pgmap v117: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:04:55.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:55 smithi188 ceph-mon[36206]: pgmap v118: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:55.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:55 smithi019 ceph-mon[35944]: pgmap v118: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:55.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:55 smithi017 ceph-mon[31926]: pgmap v118: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:57.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:57 smithi188 ceph-mon[36206]: pgmap v119: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:04:57.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:57 smithi019 ceph-mon[35944]: pgmap v119: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:04:57.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:57 smithi017 ceph-mon[31926]: pgmap v119: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:04:59.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:04:59 smithi188 ceph-mon[36206]: pgmap v120: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:59.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:04:59 smithi019 ceph-mon[35944]: pgmap v120: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:04:59.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:04:59 smithi017 ceph-mon[31926]: pgmap v120: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:01.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:01 smithi188 ceph-mon[36206]: pgmap v121: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:05:01.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:01 smithi019 ceph-mon[35944]: pgmap v121: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:05:01.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:01 smithi017 ceph-mon[31926]: pgmap v121: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:05:03.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:03 smithi188 ceph-mon[36206]: pgmap v122: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:03.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:03 smithi019 ceph-mon[35944]: pgmap v122: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:03.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:03 smithi017 ceph-mon[31926]: pgmap v122: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:05.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:05 smithi188 ceph-mon[36206]: pgmap v123: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:05.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:05 smithi019 ceph-mon[35944]: pgmap v123: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:05 smithi017 ceph-mon[31926]: pgmap v123: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:07.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:07 smithi188 ceph-mon[36206]: pgmap v124: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:05:07.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:07 smithi019 ceph-mon[35944]: pgmap v124: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:05:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:07 smithi017 ceph-mon[31926]: pgmap v124: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:05:09.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:09 smithi188 ceph-mon[36206]: pgmap v125: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:05:09.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:09 smithi019 ceph-mon[35944]: pgmap v125: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:05:09.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:09 smithi017 ceph-mon[31926]: pgmap v125: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:05:11.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:11 smithi188 ceph-mon[36206]: pgmap v126: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-24T03:05:11.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:11 smithi019 ceph-mon[35944]: pgmap v126: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-24T03:05:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:11 smithi017 ceph-mon[31926]: pgmap v126: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-24T03:05:13.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:13 smithi188 ceph-mon[36206]: pgmap v127: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-24T03:05:13.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:13 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:05:13.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:13 smithi019 ceph-mon[35944]: pgmap v127: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-24T03:05:13.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:13 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:05:13.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:13 smithi017 ceph-mon[31926]: pgmap v127: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-24T03:05:13.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:13 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:05:14.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:14 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:05:14.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:14 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:05:14.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:14 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:05:14.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:14 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:05:14.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:14 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:05:14.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:14 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:05:14.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:14 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:05:14.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:14 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:05:14.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:14 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:05:16.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:15 smithi188 ceph-mon[36206]: pgmap v128: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:05:16.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:15 smithi019 ceph-mon[35944]: pgmap v128: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:05:16.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:15 smithi017 ceph-mon[31926]: pgmap v128: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:05:18.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:17 smithi188 ceph-mon[36206]: pgmap v129: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:05:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:17 smithi019 ceph-mon[35944]: pgmap v129: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:05:18.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:17 smithi017 ceph-mon[31926]: pgmap v129: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:05:20.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:19 smithi188 ceph-mon[36206]: pgmap v130: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-24T03:05:20.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:19 smithi019 ceph-mon[35944]: pgmap v130: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-24T03:05:20.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:19 smithi017 ceph-mon[31926]: pgmap v130: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-24T03:05:22.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:21 smithi188 ceph-mon[36206]: pgmap v131: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-24T03:05:22.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:21 smithi019 ceph-mon[35944]: pgmap v131: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-24T03:05:22.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:21 smithi017 ceph-mon[31926]: pgmap v131: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-24T03:05:24.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:23 smithi188 ceph-mon[36206]: pgmap v132: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:24.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:23 smithi019 ceph-mon[35944]: pgmap v132: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:24.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:23 smithi017 ceph-mon[31926]: pgmap v132: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:26.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:25 smithi188 ceph-mon[36206]: pgmap v133: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:26.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:25 smithi019 ceph-mon[35944]: pgmap v133: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:26.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:25 smithi017 ceph-mon[31926]: pgmap v133: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:28.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:27 smithi188 ceph-mon[36206]: pgmap v134: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:05:28.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:27 smithi019 ceph-mon[35944]: pgmap v134: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:05:28.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:27 smithi017 ceph-mon[31926]: pgmap v134: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:05:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:29 smithi188 ceph-mon[36206]: pgmap v135: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:29 smithi019 ceph-mon[35944]: pgmap v135: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:30.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:29 smithi017 ceph-mon[31926]: pgmap v135: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:31 smithi188 ceph-mon[36206]: pgmap v136: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:05:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:31 smithi019 ceph-mon[35944]: pgmap v136: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:05:32.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:31 smithi017 ceph-mon[31926]: pgmap v136: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:05:34.258 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:33 smithi017 ceph-mon[31926]: pgmap v137: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:34.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:33 smithi188 ceph-mon[36206]: pgmap v137: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:34.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:33 smithi019 ceph-mon[35944]: pgmap v137: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:36.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:35 smithi188 ceph-mon[36206]: pgmap v138: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:35 smithi019 ceph-mon[35944]: pgmap v138: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:36.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:35 smithi017 ceph-mon[31926]: pgmap v138: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:38.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:37 smithi188 ceph-mon[36206]: pgmap v139: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:05:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:37 smithi019 ceph-mon[35944]: pgmap v139: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:05:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:37 smithi017 ceph-mon[31926]: pgmap v139: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:05:40.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:39 smithi188 ceph-mon[36206]: pgmap v140: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:39 smithi019 ceph-mon[35944]: pgmap v140: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:40.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:39 smithi017 ceph-mon[31926]: pgmap v140: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:42.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:42 smithi188 ceph-mon[36206]: pgmap v141: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:05:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:42 smithi019 ceph-mon[35944]: pgmap v141: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:05:42.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:42 smithi017 ceph-mon[31926]: pgmap v141: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:05:44.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:44 smithi188 ceph-mon[36206]: pgmap v142: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:44 smithi019 ceph-mon[35944]: pgmap v142: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:44.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:44 smithi017 ceph-mon[31926]: pgmap v142: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:46.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:46 smithi188 ceph-mon[36206]: pgmap v143: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:46.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:46 smithi019 ceph-mon[35944]: pgmap v143: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:46.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:46 smithi017 ceph-mon[31926]: pgmap v143: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:48.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:48 smithi188 ceph-mon[36206]: pgmap v144: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:05:48.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:48 smithi019 ceph-mon[35944]: pgmap v144: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:05:48.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:48 smithi017 ceph-mon[31926]: pgmap v144: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:05:50.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:50 smithi188 ceph-mon[36206]: pgmap v145: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:50 smithi019 ceph-mon[35944]: pgmap v145: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:50 smithi017 ceph-mon[31926]: pgmap v145: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:52.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:52 smithi188 ceph-mon[36206]: pgmap v146: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:05:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:52 smithi019 ceph-mon[35944]: pgmap v146: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:05:52.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:52 smithi017 ceph-mon[31926]: pgmap v146: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:05:54.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:54 smithi188 ceph-mon[36206]: pgmap v147: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:54.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:54 smithi019 ceph-mon[35944]: pgmap v147: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:54.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:54 smithi017 ceph-mon[31926]: pgmap v147: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:05:56.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:56 smithi188 ceph-mon[36206]: pgmap v148: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:56 smithi019 ceph-mon[35944]: pgmap v148: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:56.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:56 smithi017 ceph-mon[31926]: pgmap v148: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:05:57.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:57 smithi188 ceph-mon[36206]: evicting unresponsive client smithi017:x (15552), after 300.353 seconds 2024-03-24T03:05:57.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:57 smithi188 ceph-mon[36206]: Evicting (and blocklisting) client session 15552 (172.21.15.17:0/1112257866) 2024-03-24T03:05:57.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:57 smithi188 ceph-mon[36206]: from='mds.? [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624]' entity='mds.c' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.17:0/1112257866"}]: dispatch 2024-03-24T03:05:57.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:57 smithi188 ceph-mon[36206]: from='mds.? ' entity='mds.c' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.17:0/1112257866"}]: dispatch 2024-03-24T03:05:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:57 smithi019 ceph-mon[35944]: evicting unresponsive client smithi017:x (15552), after 300.353 seconds 2024-03-24T03:05:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:57 smithi019 ceph-mon[35944]: Evicting (and blocklisting) client session 15552 (172.21.15.17:0/1112257866) 2024-03-24T03:05:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:57 smithi019 ceph-mon[35944]: from='mds.? [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624]' entity='mds.c' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.17:0/1112257866"}]: dispatch 2024-03-24T03:05:57.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:57 smithi019 ceph-mon[35944]: from='mds.? ' entity='mds.c' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.17:0/1112257866"}]: dispatch 2024-03-24T03:05:57.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:57 smithi017 ceph-mon[31926]: evicting unresponsive client smithi017:x (15552), after 300.353 seconds 2024-03-24T03:05:57.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:57 smithi017 ceph-mon[31926]: Evicting (and blocklisting) client session 15552 (172.21.15.17:0/1112257866) 2024-03-24T03:05:57.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:57 smithi017 ceph-mon[31926]: from='mds.? [v2:172.21.15.188:6836/3512350624,v1:172.21.15.188:6837/3512350624]' entity='mds.c' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.17:0/1112257866"}]: dispatch 2024-03-24T03:05:57.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:57 smithi017 ceph-mon[31926]: from='mds.? ' entity='mds.c' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.17:0/1112257866"}]: dispatch 2024-03-24T03:05:58.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:58 smithi188 ceph-mon[36206]: pgmap v149: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:05:58.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:58 smithi188 ceph-mon[36206]: from='mds.? ' entity='mds.c' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.17:0/1112257866"}]': finished 2024-03-24T03:05:58.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:05:58 smithi188 ceph-mon[36206]: osdmap e95: 12 total, 12 up, 12 in 2024-03-24T03:05:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:58 smithi019 ceph-mon[35944]: pgmap v149: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:05:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:58 smithi019 ceph-mon[35944]: from='mds.? ' entity='mds.c' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.17:0/1112257866"}]': finished 2024-03-24T03:05:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:05:58 smithi019 ceph-mon[35944]: osdmap e95: 12 total, 12 up, 12 in 2024-03-24T03:05:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:58 smithi017 ceph-mon[31926]: pgmap v149: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:05:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:58 smithi017 ceph-mon[31926]: from='mds.? ' entity='mds.c' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.17:0/1112257866"}]': finished 2024-03-24T03:05:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:05:58 smithi017 ceph-mon[31926]: osdmap e95: 12 total, 12 up, 12 in 2024-03-24T03:06:00.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:00 smithi188 ceph-mon[36206]: pgmap v151: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-24T03:06:00.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:00 smithi019 ceph-mon[35944]: pgmap v151: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-24T03:06:00.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:00 smithi017 ceph-mon[31926]: pgmap v151: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-24T03:06:02.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:02 smithi188 ceph-mon[36206]: pgmap v152: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-24T03:06:02.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:02 smithi019 ceph-mon[35944]: pgmap v152: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-24T03:06:02.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:02 smithi017 ceph-mon[31926]: pgmap v152: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-24T03:06:04.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:04 smithi188 ceph-mon[36206]: pgmap v153: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-24T03:06:04.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:04 smithi019 ceph-mon[35944]: pgmap v153: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-24T03:06:04.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:04 smithi017 ceph-mon[31926]: pgmap v153: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-24T03:06:06.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:06 smithi188 ceph-mon[36206]: pgmap v154: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-24T03:06:06.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:06 smithi019 ceph-mon[35944]: pgmap v154: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-24T03:06:06.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:06 smithi017 ceph-mon[31926]: pgmap v154: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-24T03:06:08.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:08 smithi188 ceph-mon[36206]: pgmap v155: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 28 op/s 2024-03-24T03:06:08.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:08 smithi019 ceph-mon[35944]: pgmap v155: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 28 op/s 2024-03-24T03:06:08.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:08 smithi017 ceph-mon[31926]: pgmap v155: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 28 op/s 2024-03-24T03:06:10.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:10 smithi188 ceph-mon[36206]: pgmap v156: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-03-24T03:06:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:10 smithi019 ceph-mon[35944]: pgmap v156: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-03-24T03:06:10.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:10 smithi017 ceph-mon[31926]: pgmap v156: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-03-24T03:06:12.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:12 smithi188 ceph-mon[36206]: pgmap v157: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-24T03:06:12.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:12 smithi019 ceph-mon[35944]: pgmap v157: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-24T03:06:12.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:12 smithi017 ceph-mon[31926]: pgmap v157: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-24T03:06:13.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:13 smithi017 ceph-mon[31926]: pgmap v158: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-24T03:06:13.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:13 smithi188 ceph-mon[36206]: pgmap v158: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-24T03:06:13.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:13 smithi019 ceph-mon[35944]: pgmap v158: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-24T03:06:14.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:14 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:06:14.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:14 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:06:14.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:14 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:06:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:15 smithi017 ceph-mon[31926]: pgmap v159: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:06:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:15 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:06:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:15 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:06:15.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:15 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:06:15.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:15 smithi188 ceph-mon[36206]: pgmap v159: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:06:15.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:15 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:06:15.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:15 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:06:15.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:15 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:06:15.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:15 smithi019 ceph-mon[35944]: pgmap v159: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:06:15.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:15 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:06:15.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:15 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:06:15.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:15 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:06:17.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:17 smithi188 ceph-mon[36206]: pgmap v160: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:06:17.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:17 smithi019 ceph-mon[35944]: pgmap v160: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:06:17.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:17 smithi017 ceph-mon[31926]: pgmap v160: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:06:19.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:19 smithi188 ceph-mon[36206]: pgmap v161: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-24T03:06:19.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:19 smithi019 ceph-mon[35944]: pgmap v161: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-24T03:06:19.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:19 smithi017 ceph-mon[31926]: pgmap v161: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-24T03:06:21.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:21 smithi188 ceph-mon[36206]: pgmap v162: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-24T03:06:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:21 smithi019 ceph-mon[35944]: pgmap v162: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-24T03:06:21.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:21 smithi017 ceph-mon[31926]: pgmap v162: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-24T03:06:23.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:23 smithi188 ceph-mon[36206]: pgmap v163: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:06:23.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:23 smithi019 ceph-mon[35944]: pgmap v163: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:06:23.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:23 smithi017 ceph-mon[31926]: pgmap v163: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:06:25.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:25 smithi188 ceph-mon[36206]: pgmap v164: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:25.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:25 smithi019 ceph-mon[35944]: pgmap v164: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:25.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:25 smithi017 ceph-mon[31926]: pgmap v164: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:27.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:27 smithi188 ceph-mon[36206]: pgmap v165: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:06:27.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:27 smithi019 ceph-mon[35944]: pgmap v165: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:06:27.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:27 smithi017 ceph-mon[31926]: pgmap v165: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:06:29.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:29 smithi188 ceph-mon[36206]: pgmap v166: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:29.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:29 smithi019 ceph-mon[35944]: pgmap v166: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:29.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:29 smithi017 ceph-mon[31926]: pgmap v166: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:31.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:31 smithi188 ceph-mon[36206]: pgmap v167: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:06:31.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:31 smithi019 ceph-mon[35944]: pgmap v167: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:06:31.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:31 smithi017 ceph-mon[31926]: pgmap v167: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:06:33.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:33 smithi188 ceph-mon[36206]: pgmap v168: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:06:33.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:33 smithi019 ceph-mon[35944]: pgmap v168: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:06:33.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:33 smithi017 ceph-mon[31926]: pgmap v168: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:06:35.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:35 smithi188 ceph-mon[36206]: pgmap v169: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:35.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:35 smithi019 ceph-mon[35944]: pgmap v169: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:35.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:35 smithi017 ceph-mon[31926]: pgmap v169: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:37.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:37 smithi188 ceph-mon[36206]: pgmap v170: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:06:37.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:37 smithi019 ceph-mon[35944]: pgmap v170: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:06:37.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:37 smithi017 ceph-mon[31926]: pgmap v170: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:06:39.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:39 smithi188 ceph-mon[36206]: pgmap v171: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:39.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:39 smithi019 ceph-mon[35944]: pgmap v171: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:39.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:39 smithi017 ceph-mon[31926]: pgmap v171: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:41.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:41 smithi188 ceph-mon[36206]: pgmap v172: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:06:41.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:41 smithi019 ceph-mon[35944]: pgmap v172: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:06:41.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:41 smithi017 ceph-mon[31926]: pgmap v172: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:06:43.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:43 smithi188 ceph-mon[36206]: pgmap v173: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:06:43.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:43 smithi019 ceph-mon[35944]: pgmap v173: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:06:43.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:43 smithi017 ceph-mon[31926]: pgmap v173: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:06:45.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:45 smithi188 ceph-mon[36206]: pgmap v174: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:45.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:45 smithi019 ceph-mon[35944]: pgmap v174: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:45.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:45 smithi017 ceph-mon[31926]: pgmap v174: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:47.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:47 smithi188 ceph-mon[36206]: pgmap v175: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:06:47.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:47 smithi019 ceph-mon[35944]: pgmap v175: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:06:47.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:47 smithi017 ceph-mon[31926]: pgmap v175: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:06:49.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:49 smithi188 ceph-mon[36206]: pgmap v176: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:49.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:49 smithi019 ceph-mon[35944]: pgmap v176: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:49.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:49 smithi017 ceph-mon[31926]: pgmap v176: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:51.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:51 smithi188 ceph-mon[36206]: pgmap v177: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:06:51.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:51 smithi019 ceph-mon[35944]: pgmap v177: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:06:51.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:51 smithi017 ceph-mon[31926]: pgmap v177: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:06:53.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:53 smithi188 ceph-mon[36206]: pgmap v178: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:06:53.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:53 smithi019 ceph-mon[35944]: pgmap v178: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:06:53.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:53 smithi017 ceph-mon[31926]: pgmap v178: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:06:55.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:55 smithi188 ceph-mon[36206]: pgmap v179: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:55.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:55 smithi019 ceph-mon[35944]: pgmap v179: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:55.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:55 smithi017 ceph-mon[31926]: pgmap v179: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:57.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:57 smithi188 ceph-mon[36206]: pgmap v180: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:06:57.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:57 smithi019 ceph-mon[35944]: pgmap v180: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:06:57.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:57 smithi017 ceph-mon[31926]: pgmap v180: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:06:59.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:06:59 smithi188 ceph-mon[36206]: pgmap v181: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:59.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:06:59 smithi019 ceph-mon[35944]: pgmap v181: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:06:59.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:06:59 smithi017 ceph-mon[31926]: pgmap v181: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:01.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:01 smithi188 ceph-mon[36206]: pgmap v182: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:07:01.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:01 smithi019 ceph-mon[35944]: pgmap v182: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:07:01.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:01 smithi017 ceph-mon[31926]: pgmap v182: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:07:03.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:03 smithi188 ceph-mon[36206]: pgmap v183: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:03.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:03 smithi019 ceph-mon[35944]: pgmap v183: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:03.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:03 smithi017 ceph-mon[31926]: pgmap v183: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:05.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:05 smithi188 ceph-mon[36206]: pgmap v184: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:05.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:05 smithi019 ceph-mon[35944]: pgmap v184: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:05.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:05 smithi017 ceph-mon[31926]: pgmap v184: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:07.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:07 smithi188 ceph-mon[36206]: pgmap v185: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:07:07.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:07 smithi019 ceph-mon[35944]: pgmap v185: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:07:07.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:07 smithi017 ceph-mon[31926]: pgmap v185: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:07:09.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:09 smithi188 ceph-mon[36206]: pgmap v186: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:07:09.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:09 smithi019 ceph-mon[35944]: pgmap v186: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:07:09.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:09 smithi017 ceph-mon[31926]: pgmap v186: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:07:11.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:11 smithi188 ceph-mon[36206]: pgmap v187: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-24T03:07:11.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:11 smithi019 ceph-mon[35944]: pgmap v187: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-24T03:07:11.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:11 smithi017 ceph-mon[31926]: pgmap v187: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-24T03:07:13.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:13 smithi188 ceph-mon[36206]: pgmap v188: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-24T03:07:13.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:13 smithi019 ceph-mon[35944]: pgmap v188: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-24T03:07:13.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:13 smithi017 ceph-mon[31926]: pgmap v188: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-24T03:07:15.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:15 smithi188 ceph-mon[36206]: pgmap v189: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:07:15.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:15 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:07:15.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:15 smithi019 ceph-mon[35944]: pgmap v189: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:07:15.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:15 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:07:15.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:15 smithi017 ceph-mon[31926]: pgmap v189: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-24T03:07:15.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:15 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:07:16.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:16 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:07:16.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:16 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:07:16.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:16 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:07:16.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:16 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:07:16.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:16 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:07:16.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:16 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:07:16.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:16 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:07:16.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:16 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:07:16.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:16 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:07:18.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:17 smithi188 ceph-mon[36206]: pgmap v190: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:07:18.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:17 smithi019 ceph-mon[35944]: pgmap v190: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:07:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:17 smithi017 ceph-mon[31926]: pgmap v190: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-24T03:07:20.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:19 smithi188 ceph-mon[36206]: pgmap v191: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-24T03:07:20.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:19 smithi019 ceph-mon[35944]: pgmap v191: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-24T03:07:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:19 smithi017 ceph-mon[31926]: pgmap v191: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-24T03:07:22.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:21 smithi188 ceph-mon[36206]: pgmap v192: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-24T03:07:22.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:21 smithi019 ceph-mon[35944]: pgmap v192: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-24T03:07:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:21 smithi017 ceph-mon[31926]: pgmap v192: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-24T03:07:24.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:23 smithi188 ceph-mon[36206]: pgmap v193: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:24.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:23 smithi019 ceph-mon[35944]: pgmap v193: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:23 smithi017 ceph-mon[31926]: pgmap v193: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:26.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:25 smithi188 ceph-mon[36206]: pgmap v194: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:26.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:25 smithi019 ceph-mon[35944]: pgmap v194: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:26.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:25 smithi017 ceph-mon[31926]: pgmap v194: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:28.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:27 smithi188 ceph-mon[36206]: pgmap v195: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:07:28.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:27 smithi019 ceph-mon[35944]: pgmap v195: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:07:28.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:27 smithi017 ceph-mon[31926]: pgmap v195: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:07:30.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:29 smithi188 ceph-mon[36206]: pgmap v196: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:30.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:29 smithi019 ceph-mon[35944]: pgmap v196: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:30.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:29 smithi017 ceph-mon[31926]: pgmap v196: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:32.050 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:31 smithi188 ceph-mon[36206]: pgmap v197: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:07:32.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:31 smithi019 ceph-mon[35944]: pgmap v197: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:07:32.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:31 smithi017 ceph-mon[31926]: pgmap v197: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:07:34.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:33 smithi188 ceph-mon[36206]: pgmap v198: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:34.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:33 smithi019 ceph-mon[35944]: pgmap v198: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:33 smithi017 ceph-mon[31926]: pgmap v198: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:36.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:35 smithi188 ceph-mon[36206]: pgmap v199: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:36.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:35 smithi019 ceph-mon[35944]: pgmap v199: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:35 smithi017 ceph-mon[31926]: pgmap v199: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:38.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:37 smithi188 ceph-mon[36206]: pgmap v200: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:07:38.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:37 smithi019 ceph-mon[35944]: pgmap v200: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:07:38.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:37 smithi017 ceph-mon[31926]: pgmap v200: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:07:40.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:39 smithi188 ceph-mon[36206]: pgmap v201: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:40.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:39 smithi019 ceph-mon[35944]: pgmap v201: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:40.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:39 smithi017 ceph-mon[31926]: pgmap v201: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:42.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:41 smithi188 ceph-mon[36206]: pgmap v202: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:07:42.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:41 smithi019 ceph-mon[35944]: pgmap v202: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:07:42.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:41 smithi017 ceph-mon[31926]: pgmap v202: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:07:44.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:43 smithi188 ceph-mon[36206]: pgmap v203: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:44.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:43 smithi019 ceph-mon[35944]: pgmap v203: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:44.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:43 smithi017 ceph-mon[31926]: pgmap v203: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:46.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:45 smithi188 ceph-mon[36206]: pgmap v204: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:46.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:45 smithi019 ceph-mon[35944]: pgmap v204: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:46.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:45 smithi017 ceph-mon[31926]: pgmap v204: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:48.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:47 smithi188 ceph-mon[36206]: pgmap v205: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:07:48.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:47 smithi019 ceph-mon[35944]: pgmap v205: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:07:48.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:47 smithi017 ceph-mon[31926]: pgmap v205: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:07:50.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:49 smithi188 ceph-mon[36206]: pgmap v206: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:50.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:49 smithi019 ceph-mon[35944]: pgmap v206: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:50.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:49 smithi017 ceph-mon[31926]: pgmap v206: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:52.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:51 smithi019 ceph-mon[35944]: pgmap v207: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:07:52.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:51 smithi017 ceph-mon[31926]: pgmap v207: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:07:52.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:51 smithi188 ceph-mon[36206]: pgmap v207: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:07:54.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:53 smithi019 ceph-mon[35944]: pgmap v208: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:54.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:53 smithi017 ceph-mon[31926]: pgmap v208: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:54.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:53 smithi188 ceph-mon[36206]: pgmap v208: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:07:56.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:55 smithi019 ceph-mon[35944]: pgmap v209: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:56.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:55 smithi017 ceph-mon[31926]: pgmap v209: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:56.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:55 smithi188 ceph-mon[36206]: pgmap v209: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:07:58.071 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:57 smithi019 ceph-mon[35944]: pgmap v210: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:07:58.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:57 smithi017 ceph-mon[31926]: pgmap v210: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:07:58.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:57 smithi188 ceph-mon[36206]: pgmap v210: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:08:00.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:07:59 smithi019 ceph-mon[35944]: pgmap v211: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:07:59 smithi017 ceph-mon[31926]: pgmap v211: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:00.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:07:59 smithi188 ceph-mon[36206]: pgmap v211: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:02.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:01 smithi019 ceph-mon[35944]: pgmap v212: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-24T03:08:02.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:01 smithi017 ceph-mon[31926]: pgmap v212: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-24T03:08:02.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:01 smithi188 ceph-mon[36206]: pgmap v212: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-24T03:08:04.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:03 smithi019 ceph-mon[35944]: pgmap v213: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-24T03:08:04.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:03 smithi017 ceph-mon[31926]: pgmap v213: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-24T03:08:04.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:03 smithi188 ceph-mon[36206]: pgmap v213: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-24T03:08:06.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:05 smithi019 ceph-mon[35944]: pgmap v214: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-24T03:08:06.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:05 smithi017 ceph-mon[31926]: pgmap v214: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-24T03:08:06.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:05 smithi188 ceph-mon[36206]: pgmap v214: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-24T03:08:08.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:07 smithi019 ceph-mon[35944]: pgmap v215: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-24T03:08:08.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:07 smithi017 ceph-mon[31926]: pgmap v215: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-24T03:08:08.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:07 smithi188 ceph-mon[36206]: pgmap v215: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-24T03:08:09.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:08 smithi017 ceph-mon[31926]: osdmap e96: 12 total, 12 up, 12 in 2024-03-24T03:08:09.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:08 smithi188 ceph-mon[36206]: osdmap e96: 12 total, 12 up, 12 in 2024-03-24T03:08:09.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:08 smithi019 ceph-mon[35944]: osdmap e96: 12 total, 12 up, 12 in 2024-03-24T03:08:10.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:09 smithi017 ceph-mon[31926]: pgmap v217: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:08:10.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:09 smithi188 ceph-mon[36206]: pgmap v217: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:08:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:09 smithi019 ceph-mon[35944]: pgmap v217: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:08:12.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:11 smithi017 ceph-mon[31926]: pgmap v218: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-03-24T03:08:12.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:11 smithi188 ceph-mon[36206]: pgmap v218: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-03-24T03:08:12.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:11 smithi019 ceph-mon[35944]: pgmap v218: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-03-24T03:08:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:13 smithi017 ceph-mon[31926]: pgmap v219: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:08:14.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:13 smithi188 ceph-mon[36206]: pgmap v219: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:08:14.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:13 smithi019 ceph-mon[35944]: pgmap v219: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:08:16.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:15 smithi017 ceph-mon[31926]: pgmap v220: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:08:16.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:15 smithi017 ceph-mon[31926]: osdmap e97: 12 total, 12 up, 12 in 2024-03-24T03:08:16.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:15 smithi188 ceph-mon[36206]: pgmap v220: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:08:16.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:15 smithi188 ceph-mon[36206]: osdmap e97: 12 total, 12 up, 12 in 2024-03-24T03:08:16.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:15 smithi019 ceph-mon[35944]: pgmap v220: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:08:16.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:15 smithi019 ceph-mon[35944]: osdmap e97: 12 total, 12 up, 12 in 2024-03-24T03:08:17.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:16 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:08:17.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:16 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:08:17.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:16 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:08:17.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:16 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:08:17.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:16 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:08:17.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:16 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:08:17.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:16 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:08:17.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:16 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:08:17.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:16 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:08:17.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:16 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:08:17.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:16 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:08:17.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:16 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:08:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:17 smithi017 ceph-mon[31926]: pgmap v222: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 242 B/s wr, 12 op/s 2024-03-24T03:08:18.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:17 smithi188 ceph-mon[36206]: pgmap v222: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 242 B/s wr, 12 op/s 2024-03-24T03:08:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:17 smithi019 ceph-mon[35944]: pgmap v222: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 242 B/s wr, 12 op/s 2024-03-24T03:08:20.099 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:19 smithi188 ceph-mon[36206]: pgmap v223: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:08:20.122 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:19 smithi019 ceph-mon[35944]: pgmap v223: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:08:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:19 smithi017 ceph-mon[31926]: pgmap v223: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:08:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:21 smithi017 ceph-mon[31926]: pgmap v224: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:08:22.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:21 smithi188 ceph-mon[36206]: pgmap v224: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:08:22.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:21 smithi019 ceph-mon[35944]: pgmap v224: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:08:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:23 smithi017 ceph-mon[31926]: pgmap v225: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:08:24.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:23 smithi188 ceph-mon[36206]: pgmap v225: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:08:24.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:23 smithi019 ceph-mon[35944]: pgmap v225: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:08:26.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:25 smithi017 ceph-mon[31926]: pgmap v226: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:08:26.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:25 smithi188 ceph-mon[36206]: pgmap v226: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:08:26.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:25 smithi019 ceph-mon[35944]: pgmap v226: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:08:28.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:27 smithi017 ceph-mon[31926]: pgmap v227: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-24T03:08:28.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:27 smithi188 ceph-mon[36206]: pgmap v227: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-24T03:08:28.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:27 smithi019 ceph-mon[35944]: pgmap v227: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-24T03:08:30.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:29 smithi017 ceph-mon[31926]: pgmap v228: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:30.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:29 smithi188 ceph-mon[36206]: pgmap v228: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:29 smithi019 ceph-mon[35944]: pgmap v228: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:32.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:31 smithi017 ceph-mon[31926]: pgmap v229: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:08:32.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:31 smithi188 ceph-mon[36206]: pgmap v229: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:08:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:31 smithi019 ceph-mon[35944]: pgmap v229: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:08:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:33 smithi017 ceph-mon[31926]: pgmap v230: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:08:34.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:33 smithi188 ceph-mon[36206]: pgmap v230: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:08:34.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:33 smithi019 ceph-mon[35944]: pgmap v230: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:08:36.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:35 smithi017 ceph-mon[31926]: pgmap v231: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:36.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:35 smithi188 ceph-mon[36206]: pgmap v231: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:35 smithi019 ceph-mon[35944]: pgmap v231: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:38.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:37 smithi017 ceph-mon[31926]: pgmap v232: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:08:38.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:37 smithi188 ceph-mon[36206]: pgmap v232: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:08:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:37 smithi019 ceph-mon[35944]: pgmap v232: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:08:40.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:39 smithi017 ceph-mon[31926]: pgmap v233: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:40.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:39 smithi188 ceph-mon[36206]: pgmap v233: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:39 smithi019 ceph-mon[35944]: pgmap v233: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:42.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:41 smithi017 ceph-mon[31926]: pgmap v234: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:08:42.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:41 smithi188 ceph-mon[36206]: pgmap v234: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:08:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:41 smithi019 ceph-mon[35944]: pgmap v234: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:08:44.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:43 smithi017 ceph-mon[31926]: pgmap v235: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:08:44.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:43 smithi188 ceph-mon[36206]: pgmap v235: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:08:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:43 smithi019 ceph-mon[35944]: pgmap v235: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:08:46.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:45 smithi017 ceph-mon[31926]: pgmap v236: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:46.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:45 smithi188 ceph-mon[36206]: pgmap v236: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:46.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:45 smithi019 ceph-mon[35944]: pgmap v236: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:48.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:47 smithi017 ceph-mon[31926]: pgmap v237: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:08:48.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:47 smithi188 ceph-mon[36206]: pgmap v237: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:08:48.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:47 smithi019 ceph-mon[35944]: pgmap v237: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:08:50.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:49 smithi017 ceph-mon[31926]: pgmap v238: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:50.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:49 smithi188 ceph-mon[36206]: pgmap v238: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:49 smithi019 ceph-mon[35944]: pgmap v238: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:52.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:51 smithi017 ceph-mon[31926]: pgmap v239: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:08:52.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:51 smithi188 ceph-mon[36206]: pgmap v239: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:08:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:51 smithi019 ceph-mon[35944]: pgmap v239: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:08:54.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:53 smithi017 ceph-mon[31926]: pgmap v240: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:08:54.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:53 smithi188 ceph-mon[36206]: pgmap v240: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:08:54.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:53 smithi019 ceph-mon[35944]: pgmap v240: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-24T03:08:56.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:55 smithi017 ceph-mon[31926]: pgmap v241: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:56.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:55 smithi188 ceph-mon[36206]: pgmap v241: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:55 smithi019 ceph-mon[35944]: pgmap v241: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:08:58.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:57 smithi017 ceph-mon[31926]: pgmap v242: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:08:58.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:57 smithi188 ceph-mon[36206]: pgmap v242: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:08:58.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:57 smithi019 ceph-mon[35944]: pgmap v242: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:00.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:08:59 smithi017 ceph-mon[31926]: pgmap v243: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:00.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:08:59 smithi188 ceph-mon[36206]: pgmap v243: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:00.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:08:59 smithi019 ceph-mon[35944]: pgmap v243: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:02.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:01 smithi017 ceph-mon[31926]: pgmap v244: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-24T03:09:02.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:01 smithi188 ceph-mon[36206]: pgmap v244: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-24T03:09:02.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:01 smithi019 ceph-mon[35944]: pgmap v244: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-24T03:09:04.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:03 smithi017 ceph-mon[31926]: pgmap v245: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-03-24T03:09:04.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:03 smithi188 ceph-mon[36206]: pgmap v245: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-03-24T03:09:04.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:03 smithi019 ceph-mon[35944]: pgmap v245: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-03-24T03:09:06.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:05 smithi017 ceph-mon[31926]: pgmap v246: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-03-24T03:09:06.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:05 smithi188 ceph-mon[36206]: pgmap v246: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-03-24T03:09:06.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:05 smithi019 ceph-mon[35944]: pgmap v246: 129 pgs: 129 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-03-24T03:09:08.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:07 smithi017 ceph-mon[31926]: pgmap v247: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-24T03:09:08.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:07 smithi188 ceph-mon[36206]: pgmap v247: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-24T03:09:08.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:07 smithi019 ceph-mon[35944]: pgmap v247: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-24T03:09:09.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:08 smithi017 ceph-mon[31926]: osdmap e98: 12 total, 12 up, 12 in 2024-03-24T03:09:09.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:08 smithi188 ceph-mon[36206]: osdmap e98: 12 total, 12 up, 12 in 2024-03-24T03:09:09.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:08 smithi019 ceph-mon[35944]: osdmap e98: 12 total, 12 up, 12 in 2024-03-24T03:09:10.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:09 smithi017 ceph-mon[31926]: pgmap v249: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:09:10.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:09 smithi188 ceph-mon[36206]: pgmap v249: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:09:10.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:09 smithi019 ceph-mon[35944]: pgmap v249: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:09:12.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:11 smithi017 ceph-mon[31926]: pgmap v250: 129 pgs: 129 active+clean; 152 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-03-24T03:09:12.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:11 smithi188 ceph-mon[36206]: pgmap v250: 129 pgs: 129 active+clean; 152 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-03-24T03:09:12.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:11 smithi019 ceph-mon[35944]: pgmap v250: 129 pgs: 129 active+clean; 152 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-03-24T03:09:14.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:13 smithi017 ceph-mon[31926]: pgmap v251: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-24T03:09:14.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:13 smithi188 ceph-mon[36206]: pgmap v251: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-24T03:09:14.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:13 smithi019 ceph-mon[35944]: pgmap v251: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-24T03:09:16.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:15 smithi017 ceph-mon[31926]: pgmap v252: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-24T03:09:16.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:15 smithi017 ceph-mon[31926]: osdmap e99: 12 total, 12 up, 12 in 2024-03-24T03:09:16.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:15 smithi188 ceph-mon[36206]: pgmap v252: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-24T03:09:16.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:15 smithi188 ceph-mon[36206]: osdmap e99: 12 total, 12 up, 12 in 2024-03-24T03:09:16.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:15 smithi019 ceph-mon[35944]: pgmap v252: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-24T03:09:16.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:15 smithi019 ceph-mon[35944]: osdmap e99: 12 total, 12 up, 12 in 2024-03-24T03:09:17.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:16 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:09:17.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:16 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:09:17.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:16 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:09:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:17 smithi017 ceph-mon[31926]: pgmap v254: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 244 B/s wr, 12 op/s 2024-03-24T03:09:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:17 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:09:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:17 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:09:18.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:17 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:09:18.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:17 smithi188 ceph-mon[36206]: pgmap v254: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 244 B/s wr, 12 op/s 2024-03-24T03:09:18.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:17 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:09:18.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:17 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:09:18.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:17 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:09:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:17 smithi019 ceph-mon[35944]: pgmap v254: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 244 B/s wr, 12 op/s 2024-03-24T03:09:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:17 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:09:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:17 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:09:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:17 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:09:20.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:19 smithi017 ceph-mon[31926]: pgmap v255: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:09:20.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:19 smithi188 ceph-mon[36206]: pgmap v255: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:09:20.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:19 smithi019 ceph-mon[35944]: pgmap v255: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:09:22.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:21 smithi017 ceph-mon[31926]: pgmap v256: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:09:22.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:21 smithi188 ceph-mon[36206]: pgmap v256: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:09:22.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:21 smithi019 ceph-mon[35944]: pgmap v256: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:09:24.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:23 smithi017 ceph-mon[31926]: pgmap v257: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:09:24.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:23 smithi188 ceph-mon[36206]: pgmap v257: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:09:24.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:23 smithi019 ceph-mon[35944]: pgmap v257: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:09:26.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:25 smithi017 ceph-mon[31926]: pgmap v258: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:09:26.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:25 smithi188 ceph-mon[36206]: pgmap v258: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:09:26.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:25 smithi019 ceph-mon[35944]: pgmap v258: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:09:28.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:27 smithi017 ceph-mon[31926]: pgmap v259: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-24T03:09:28.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:27 smithi188 ceph-mon[36206]: pgmap v259: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-24T03:09:28.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:27 smithi019 ceph-mon[35944]: pgmap v259: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-24T03:09:30.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:29 smithi017 ceph-mon[31926]: pgmap v260: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:30.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:29 smithi188 ceph-mon[36206]: pgmap v260: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:29 smithi019 ceph-mon[35944]: pgmap v260: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:32.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:31 smithi017 ceph-mon[31926]: pgmap v261: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:31 smithi188 ceph-mon[36206]: pgmap v261: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:31 smithi019 ceph-mon[35944]: pgmap v261: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:34.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:33 smithi017 ceph-mon[31926]: pgmap v262: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:34.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:33 smithi188 ceph-mon[36206]: pgmap v262: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:34.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:33 smithi019 ceph-mon[35944]: pgmap v262: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:36.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:35 smithi188 ceph-mon[36206]: pgmap v263: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:35 smithi019 ceph-mon[35944]: pgmap v263: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:36.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:35 smithi017 ceph-mon[31926]: pgmap v263: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:38.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:37 smithi188 ceph-mon[36206]: pgmap v264: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:37 smithi019 ceph-mon[35944]: pgmap v264: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:37 smithi017 ceph-mon[31926]: pgmap v264: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:40.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:39 smithi188 ceph-mon[36206]: pgmap v265: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:39 smithi019 ceph-mon[35944]: pgmap v265: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:40.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:39 smithi017 ceph-mon[31926]: pgmap v265: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:42.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:41 smithi188 ceph-mon[36206]: pgmap v266: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:41 smithi019 ceph-mon[35944]: pgmap v266: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:42.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:41 smithi017 ceph-mon[31926]: pgmap v266: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:44.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:43 smithi188 ceph-mon[36206]: pgmap v267: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:43 smithi019 ceph-mon[35944]: pgmap v267: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:44.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:43 smithi017 ceph-mon[31926]: pgmap v267: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:46.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:45 smithi188 ceph-mon[36206]: pgmap v268: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:46.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:45 smithi019 ceph-mon[35944]: pgmap v268: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:46.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:45 smithi017 ceph-mon[31926]: pgmap v268: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:48.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:47 smithi188 ceph-mon[36206]: pgmap v269: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:48.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:47 smithi019 ceph-mon[35944]: pgmap v269: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:48.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:47 smithi017 ceph-mon[31926]: pgmap v269: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:50.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:49 smithi188 ceph-mon[36206]: pgmap v270: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:49 smithi019 ceph-mon[35944]: pgmap v270: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:49 smithi017 ceph-mon[31926]: pgmap v270: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:52.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:51 smithi188 ceph-mon[36206]: pgmap v271: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:51 smithi019 ceph-mon[35944]: pgmap v271: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:52.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:51 smithi017 ceph-mon[31926]: pgmap v271: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:54.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:53 smithi188 ceph-mon[36206]: pgmap v272: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:54.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:53 smithi019 ceph-mon[35944]: pgmap v272: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:54.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:53 smithi017 ceph-mon[31926]: pgmap v272: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:56.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:55 smithi188 ceph-mon[36206]: pgmap v273: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:56.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:55 smithi019 ceph-mon[35944]: pgmap v273: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:56.455 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:55 smithi017 ceph-mon[31926]: pgmap v273: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:09:58.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:57 smithi188 ceph-mon[36206]: pgmap v274: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:58.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:57 smithi019 ceph-mon[35944]: pgmap v274: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:09:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:57 smithi017 ceph-mon[31926]: pgmap v274: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:00.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:09:59 smithi188 ceph-mon[36206]: pgmap v275: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:00.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:09:59 smithi019 ceph-mon[35944]: pgmap v275: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:00.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:09:59 smithi017 ceph-mon[31926]: pgmap v275: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:01.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:00 smithi188 ceph-mon[36206]: overall HEALTH_OK 2024-03-24T03:10:01.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:00 smithi019 ceph-mon[35944]: overall HEALTH_OK 2024-03-24T03:10:01.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:00 smithi017 ceph-mon[31926]: overall HEALTH_OK 2024-03-24T03:10:02.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:01 smithi188 ceph-mon[36206]: pgmap v276: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-24T03:10:02.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:01 smithi019 ceph-mon[35944]: pgmap v276: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-24T03:10:02.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:01 smithi017 ceph-mon[31926]: pgmap v276: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-24T03:10:04.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:03 smithi188 ceph-mon[36206]: pgmap v277: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-03-24T03:10:04.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:03 smithi019 ceph-mon[35944]: pgmap v277: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-03-24T03:10:04.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:03 smithi017 ceph-mon[31926]: pgmap v277: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-03-24T03:10:06.258 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:06 smithi017 ceph-mon[31926]: pgmap v278: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-03-24T03:10:06.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:06 smithi188 ceph-mon[36206]: pgmap v278: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-03-24T03:10:06.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:06 smithi019 ceph-mon[35944]: pgmap v278: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-03-24T03:10:08.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:08 smithi188 ceph-mon[36206]: pgmap v279: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-24T03:10:08.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:08 smithi019 ceph-mon[35944]: pgmap v279: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-24T03:10:08.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:08 smithi017 ceph-mon[31926]: pgmap v279: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-24T03:10:09.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:09 smithi188 ceph-mon[36206]: osdmap e100: 12 total, 12 up, 12 in 2024-03-24T03:10:09.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:09 smithi019 ceph-mon[35944]: osdmap e100: 12 total, 12 up, 12 in 2024-03-24T03:10:09.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:09 smithi017 ceph-mon[31926]: osdmap e100: 12 total, 12 up, 12 in 2024-03-24T03:10:10.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:10 smithi019 ceph-mon[35944]: pgmap v281: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:10:10.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:10 smithi188 ceph-mon[36206]: pgmap v281: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:10:10.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:10 smithi017 ceph-mon[31926]: pgmap v281: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:10:12.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:12 smithi188 ceph-mon[36206]: pgmap v282: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:10:12.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:12 smithi019 ceph-mon[35944]: pgmap v282: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:10:12.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:12 smithi017 ceph-mon[31926]: pgmap v282: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:10:14.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:14 smithi188 ceph-mon[36206]: pgmap v283: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-03-24T03:10:14.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:14 smithi019 ceph-mon[35944]: pgmap v283: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-03-24T03:10:14.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:14 smithi017 ceph-mon[31926]: pgmap v283: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-03-24T03:10:16.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:16 smithi188 ceph-mon[36206]: pgmap v284: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-03-24T03:10:16.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:16 smithi188 ceph-mon[36206]: osdmap e101: 12 total, 12 up, 12 in 2024-03-24T03:10:16.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:16 smithi019 ceph-mon[35944]: pgmap v284: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-03-24T03:10:16.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:16 smithi019 ceph-mon[35944]: osdmap e101: 12 total, 12 up, 12 in 2024-03-24T03:10:16.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:16 smithi017 ceph-mon[31926]: pgmap v284: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-03-24T03:10:16.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:16 smithi017 ceph-mon[31926]: osdmap e101: 12 total, 12 up, 12 in 2024-03-24T03:10:18.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:18 smithi188 ceph-mon[36206]: pgmap v286: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 247 B/s wr, 12 op/s 2024-03-24T03:10:18.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:18 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:10:18.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:18 smithi019 ceph-mon[35944]: pgmap v286: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 247 B/s wr, 12 op/s 2024-03-24T03:10:18.321 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:18 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:10:18.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:18 smithi017 ceph-mon[31926]: pgmap v286: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 247 B/s wr, 12 op/s 2024-03-24T03:10:18.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:18 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:10:19.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:19 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:10:19.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:19 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:10:19.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:19 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:10:19.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:19 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:10:19.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:19 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:10:19.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:19 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:10:19.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:19 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:10:19.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:19 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:10:19.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:19 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:10:20.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:20 smithi188 ceph-mon[36206]: pgmap v287: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:10:20.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:20 smithi019 ceph-mon[35944]: pgmap v287: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:10:20.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:20 smithi017 ceph-mon[31926]: pgmap v287: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-24T03:10:22.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:22 smithi188 ceph-mon[36206]: pgmap v288: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:10:22.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:22 smithi019 ceph-mon[35944]: pgmap v288: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:10:22.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:22 smithi017 ceph-mon[31926]: pgmap v288: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:10:24.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:24 smithi188 ceph-mon[36206]: pgmap v289: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:10:24.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:24 smithi019 ceph-mon[35944]: pgmap v289: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:10:24.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:24 smithi017 ceph-mon[31926]: pgmap v289: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:10:26.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:26 smithi188 ceph-mon[36206]: pgmap v290: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:10:26.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:26 smithi019 ceph-mon[35944]: pgmap v290: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:10:26.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:26 smithi017 ceph-mon[31926]: pgmap v290: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:10:28.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:28 smithi188 ceph-mon[36206]: pgmap v291: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-24T03:10:28.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:28 smithi019 ceph-mon[35944]: pgmap v291: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-24T03:10:28.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:28 smithi017 ceph-mon[31926]: pgmap v291: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-24T03:10:30.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:30 smithi188 ceph-mon[36206]: pgmap v292: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:30.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:30 smithi019 ceph-mon[35944]: pgmap v292: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:30.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:30 smithi017 ceph-mon[31926]: pgmap v292: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:32.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:32 smithi188 ceph-mon[36206]: pgmap v293: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:32.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:32 smithi019 ceph-mon[35944]: pgmap v293: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:32.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:32 smithi017 ceph-mon[31926]: pgmap v293: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:34.297 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:34 smithi188 ceph-mon[36206]: pgmap v294: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:34.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:34 smithi019 ceph-mon[35944]: pgmap v294: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:34.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:34 smithi017 ceph-mon[31926]: pgmap v294: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:36.298 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:36 smithi188 ceph-mon[36206]: pgmap v295: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:36.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:36 smithi019 ceph-mon[35944]: pgmap v295: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:36.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:36 smithi017 ceph-mon[31926]: pgmap v295: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:38.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:38 smithi019 ceph-mon[35944]: pgmap v296: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:38.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:38 smithi017 ceph-mon[31926]: pgmap v296: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:38.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:38 smithi188 ceph-mon[36206]: pgmap v296: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:40.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:40 smithi019 ceph-mon[35944]: pgmap v297: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:40.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:40 smithi017 ceph-mon[31926]: pgmap v297: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:40.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:40 smithi188 ceph-mon[36206]: pgmap v297: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:42.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:42 smithi019 ceph-mon[35944]: pgmap v298: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:42.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:42 smithi017 ceph-mon[31926]: pgmap v298: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:42.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:42 smithi188 ceph-mon[36206]: pgmap v298: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:44.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:44 smithi019 ceph-mon[35944]: pgmap v299: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:44.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:44 smithi017 ceph-mon[31926]: pgmap v299: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:44.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:44 smithi188 ceph-mon[36206]: pgmap v299: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:46.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:46 smithi019 ceph-mon[35944]: pgmap v300: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:46.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:46 smithi017 ceph-mon[31926]: pgmap v300: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:46.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:46 smithi188 ceph-mon[36206]: pgmap v300: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:48.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:48 smithi019 ceph-mon[35944]: pgmap v301: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:48.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:48 smithi017 ceph-mon[31926]: pgmap v301: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:48.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:48 smithi188 ceph-mon[36206]: pgmap v301: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:50.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:50 smithi019 ceph-mon[35944]: pgmap v302: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:50.349 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:50 smithi188 ceph-mon[36206]: pgmap v302: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:50.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:50 smithi017 ceph-mon[31926]: pgmap v302: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:10:52.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:52 smithi019 ceph-mon[35944]: pgmap v303: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:10:52.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:52 smithi017 ceph-mon[31926]: pgmap v303: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:10:52.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:52 smithi188 ceph-mon[36206]: pgmap v303: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:10:54.320 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:54 smithi019 ceph-mon[35944]: pgmap v304: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:10:54.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:54 smithi017 ceph-mon[31926]: pgmap v304: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:10:54.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:54 smithi188 ceph-mon[36206]: pgmap v304: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:10:56.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:56 smithi017 ceph-mon[31926]: pgmap v305: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:10:56.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:56 smithi188 ceph-mon[36206]: pgmap v305: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:10:56.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:56 smithi019 ceph-mon[35944]: pgmap v305: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:10:58.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:10:58 smithi017 ceph-mon[31926]: pgmap v306: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:58.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:10:58 smithi188 ceph-mon[36206]: pgmap v306: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:10:58.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:10:58 smithi019 ceph-mon[35944]: pgmap v306: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:00.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:00 smithi017 ceph-mon[31926]: pgmap v307: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:11:00.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:00 smithi188 ceph-mon[36206]: pgmap v307: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:11:00.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:00 smithi019 ceph-mon[35944]: pgmap v307: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-24T03:11:02.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:02 smithi017 ceph-mon[31926]: pgmap v308: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:11:02.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:02 smithi188 ceph-mon[36206]: pgmap v308: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:11:02.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:02 smithi019 ceph-mon[35944]: pgmap v308: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-24T03:11:04.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:04 smithi017 ceph-mon[31926]: pgmap v309: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-03-24T03:11:04.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:04 smithi188 ceph-mon[36206]: pgmap v309: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-03-24T03:11:04.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:04 smithi019 ceph-mon[35944]: pgmap v309: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-03-24T03:11:06.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:06 smithi017 ceph-mon[31926]: pgmap v310: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-03-24T03:11:06.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:06 smithi188 ceph-mon[36206]: pgmap v310: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-03-24T03:11:06.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:06 smithi019 ceph-mon[35944]: pgmap v310: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-03-24T03:11:07.006 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-24T03:11:07.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.122+0000 7f1d1edc4640 1 -- 172.21.15.17:0/1253438088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d18105520 msgr2=0x7f1d18105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.122+0000 7f1d1edc4640 1 --2- 172.21.15.17:0/1253438088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d18105520 0x7f1d18105900 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009900 tx=0x7f1d0802bc30 comp rx=0 tx=0).stop 2024-03-24T03:11:07.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.122+0000 7f1d1edc4640 1 -- 172.21.15.17:0/1253438088 shutdown_connections 2024-03-24T03:11:07.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.122+0000 7f1d1edc4640 1 --2- 172.21.15.17:0/1253438088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d18101190 0x7f1d1810d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.122+0000 7f1d1edc4640 1 --2- 172.21.15.17:0/1253438088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d181007f0 0x7f1d18100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.122+0000 7f1d1edc4640 1 --2- 172.21.15.17:0/1253438088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d18105520 0x7f1d18105900 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.122+0000 7f1d1edc4640 1 -- 172.21.15.17:0/1253438088 >> 172.21.15.17:0/1253438088 conn(0x7f1d18077370 msgr2=0x7f1d18077780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:07.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.122+0000 7f1d1edc4640 1 -- 172.21.15.17:0/1253438088 shutdown_connections 2024-03-24T03:11:07.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.122+0000 7f1d1edc4640 1 -- 172.21.15.17:0/1253438088 wait complete. 2024-03-24T03:11:07.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d1edc4640 1 Processor -- start 2024-03-24T03:11:07.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d1edc4640 1 -- start start 2024-03-24T03:11:07.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d1edc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d181007f0 0x7f1d18197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:07.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d1edc4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d18101190 0x7f1d181984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:07.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d1edc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d18105520 0x7f1d1819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d1edc4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d18110a30 con 0x7f1d18101190 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d1edc4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d181108b0 con 0x7f1d181007f0 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d1edc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d18110bb0 con 0x7f1d18105520 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d17fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d18101190 0x7f1d181984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d1cb60640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d181007f0 0x7f1d18197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d1cb60640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d181007f0 0x7f1d18197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:35972/0 (socket says 172.21.15.17:35972) 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d17fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d18101190 0x7f1d181984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44030/0 (socket says 172.21.15.17:44030) 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d1cb60640 1 -- 172.21.15.17:0/49623781 learned_addr learned my addr 172.21.15.17:0/49623781 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.123+0000 7f1d1d361640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d18105520 0x7f1d1819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d1cb60640 1 -- 172.21.15.17:0/49623781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d18105520 msgr2=0x7f1d1819c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d1cb60640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d18105520 0x7f1d1819c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d1cb60640 1 -- 172.21.15.17:0/49623781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d18101190 msgr2=0x7f1d181984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d1cb60640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d18101190 0x7f1d181984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d1cb60640 1 -- 172.21.15.17:0/49623781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1819cf60 con 0x7f1d181007f0 2024-03-24T03:11:07.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d17fff640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d18101190 0x7f1d181984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:11:07.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d1d361640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d18105520 0x7f1d1819c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:11:07.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d1cb60640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d181007f0 0x7f1d18197f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d08006010 tx=0x7f1d0802cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:07.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d15ffb640 1 -- 172.21.15.17:0/49623781 <== mon.2 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08041070 con 0x7f1d181007f0 2024-03-24T03:11:07.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d1edc4640 1 -- 172.21.15.17:0/49623781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1819d1f0 con 0x7f1d181007f0 2024-03-24T03:11:07.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d15ffb640 1 -- 172.21.15.17:0/49623781 <== mon.2 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d08004450 con 0x7f1d181007f0 2024-03-24T03:11:07.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d15ffb640 1 -- 172.21.15.17:0/49623781 <== mon.2 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08003d00 con 0x7f1d181007f0 2024-03-24T03:11:07.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.124+0000 7f1d1edc4640 1 -- 172.21.15.17:0/49623781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d181a4ae0 con 0x7f1d181007f0 2024-03-24T03:11:07.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.125+0000 7f1d1edc4640 1 -- 172.21.15.17:0/49623781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d18101cb0 con 0x7f1d181007f0 2024-03-24T03:11:07.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.126+0000 7f1d15ffb640 1 -- 172.21.15.17:0/49623781 <== mon.2 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1d08044070 con 0x7f1d181007f0 2024-03-24T03:11:07.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.127+0000 7f1d15ffb640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f1d000a2a10 0x7f1d000a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:07.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.127+0000 7f1d17fff640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f1d000a2a10 0x7f1d000a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:07.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.127+0000 7f1d15ffb640 1 -- 172.21.15.17:0/49623781 <== mon.2 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9708+0+0 (secure 0 0 0) 0x7f1d080f0d20 con 0x7f1d181007f0 2024-03-24T03:11:07.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.127+0000 7f1d17fff640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f1d000a2a10 0x7f1d000a4ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1d18104e60 tx=0x7f1d0c005f90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:07.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.134+0000 7f1d15ffb640 1 -- 172.21.15.17:0/49623781 <== mon.2 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1d080bd690 con 0x7f1d181007f0 2024-03-24T03:11:07.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.338+0000 7f1d1edc4640 1 -- 172.21.15.17:0/49623781 --> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d18061ae0 con 0x7f1d000a2a10 2024-03-24T03:11:07.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.345+0000 7f1d15ffb640 1 -- 172.21.15.17:0/49623781 <== mgr.25334 v2:172.21.15.17:6800/1006945247 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7f1d18061ae0 con 0x7f1d000a2a10 2024-03-24T03:11:07.346 INFO:teuthology.orchestra.run.smithi017.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-24T00:00:00", "created": "2024-03-24T03:01:05", "first": "2024-03-24T03:02:00", "last": "2024-03-24T03:11:00", "last_pruned": "2024-03-24T03:11:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-03-24T03:11:07.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.347+0000 7f1d1edc4640 1 -- 172.21.15.17:0/49623781 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f1d000a2a10 msgr2=0x7f1d000a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.347+0000 7f1d1edc4640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f1d000a2a10 0x7f1d000a4ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1d18104e60 tx=0x7f1d0c005f90 comp rx=0 tx=0).stop 2024-03-24T03:11:07.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.347+0000 7f1d1edc4640 1 -- 172.21.15.17:0/49623781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d181007f0 msgr2=0x7f1d18197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.347+0000 7f1d1edc4640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d181007f0 0x7f1d18197f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d08006010 tx=0x7f1d0802cb30 comp rx=0 tx=0).stop 2024-03-24T03:11:07.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.347+0000 7f1d1edc4640 1 -- 172.21.15.17:0/49623781 shutdown_connections 2024-03-24T03:11:07.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.347+0000 7f1d1edc4640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f1d000a2a10 0x7f1d000a4ed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.347+0000 7f1d1edc4640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d18105520 0x7f1d1819c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.347+0000 7f1d1edc4640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d18101190 0x7f1d181984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.347+0000 7f1d1edc4640 1 --2- 172.21.15.17:0/49623781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d181007f0 0x7f1d18197f70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.347+0000 7f1d1edc4640 1 -- 172.21.15.17:0/49623781 >> 172.21.15.17:0/49623781 conn(0x7f1d18077370 msgr2=0x7f1d1806a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:07.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.347+0000 7f1d1edc4640 1 -- 172.21.15.17:0/49623781 shutdown_connections 2024-03-24T03:11:07.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.347+0000 7f1d1edc4640 1 -- 172.21.15.17:0/49623781 wait complete. 2024-03-24T03:11:07.358 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-24T03:11:07.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.464+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3499783482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0181025c0 msgr2=0x7fc0180ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.464+0000 7fc01e8f1640 1 --2- 172.21.15.17:0/3499783482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0181025c0 0x7fc0180ffba0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc008009900 tx=0x7fc00802bcd0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.464+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3499783482 shutdown_connections 2024-03-24T03:11:07.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.464+0000 7fc01e8f1640 1 --2- 172.21.15.17:0/3499783482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018100700 0x7fc018110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.464+0000 7fc01e8f1640 1 --2- 172.21.15.17:0/3499783482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0181029a0 0x7fc0181000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.464+0000 7fc01e8f1640 1 --2- 172.21.15.17:0/3499783482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0181025c0 0x7fc0180ffba0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.464+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3499783482 >> 172.21.15.17:0/3499783482 conn(0x7fc018076af0 msgr2=0x7fc0180fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:07.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.464+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3499783482 shutdown_connections 2024-03-24T03:11:07.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.464+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3499783482 wait complete. 2024-03-24T03:11:07.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01e8f1640 1 Processor -- start 2024-03-24T03:11:07.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01e8f1640 1 -- start start 2024-03-24T03:11:07.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01e8f1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018100700 0x7fc01819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:07.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01e8f1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0181025c0 0x7fc01819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:07.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01e8f1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0181029a0 0x7fc01819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:07.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01e8f1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc018113c70 con 0x7fc0181029a0 2024-03-24T03:11:07.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01e8f1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc018113af0 con 0x7fc018100700 2024-03-24T03:11:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01e8f1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc018113df0 con 0x7fc0181025c0 2024-03-24T03:11:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01ce8e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0181029a0 0x7fc01819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01ce8e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0181029a0 0x7fc01819e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44036/0 (socket says 172.21.15.17:44036) 2024-03-24T03:11:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01ce8e640 1 -- 172.21.15.17:0/3236352310 learned_addr learned my addr 172.21.15.17:0/3236352310 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc017fff640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018100700 0x7fc01819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc0177fe640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0181025c0 0x7fc01819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01ce8e640 1 -- 172.21.15.17:0/3236352310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0181025c0 msgr2=0x7fc01819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.465+0000 7fc01ce8e640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0181025c0 0x7fc01819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.466+0000 7fc01ce8e640 1 -- 172.21.15.17:0/3236352310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018100700 msgr2=0x7fc01819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.466+0000 7fc01ce8e640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018100700 0x7fc01819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.466+0000 7fc01ce8e640 1 -- 172.21.15.17:0/3236352310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01819f0e0 con 0x7fc0181029a0 2024-03-24T03:11:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.466+0000 7fc01ce8e640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0181029a0 0x7fc01819e9b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc01000eb10 tx=0x7fc01000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:07.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.466+0000 7fc0157fa640 1 -- 172.21.15.17:0/3236352310 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010004b20 con 0x7fc0181029a0 2024-03-24T03:11:07.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.466+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3236352310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01819f3d0 con 0x7fc0181029a0 2024-03-24T03:11:07.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.466+0000 7fc0157fa640 1 -- 172.21.15.17:0/3236352310 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc010004cc0 con 0x7fc0181029a0 2024-03-24T03:11:07.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.466+0000 7fc0157fa640 1 -- 172.21.15.17:0/3236352310 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010010760 con 0x7fc0181029a0 2024-03-24T03:11:07.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.466+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3236352310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0181a6c60 con 0x7fc0181029a0 2024-03-24T03:11:07.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.467+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3236352310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01804f400 con 0x7fc0181029a0 2024-03-24T03:11:07.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.468+0000 7fc0157fa640 1 -- 172.21.15.17:0/3236352310 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc010005020 con 0x7fc0181029a0 2024-03-24T03:11:07.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.468+0000 7fc0157fa640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc0000a2a10 0x7fc0000a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:07.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.469+0000 7fc017fff640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc0000a2a10 0x7fc0000a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:07.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.469+0000 7fc0157fa640 1 -- 172.21.15.17:0/3236352310 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9708+0+0 (secure 0 0 0) 0x7fc0100cf880 con 0x7fc0181029a0 2024-03-24T03:11:07.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.469+0000 7fc017fff640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc0000a2a10 0x7fc0000a4ed0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc00802efd0 tx=0x7fc00802cf90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:07.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.473+0000 7fc0157fa640 1 -- 172.21.15.17:0/3236352310 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc01009c140 con 0x7fc0181029a0 2024-03-24T03:11:07.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.678+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3236352310 --> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc018061ae0 con 0x7fc0000a2a10 2024-03-24T03:11:07.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.684+0000 7fc0157fa640 1 -- 172.21.15.17:0/3236352310 <== mgr.25334 v2:172.21.15.17:6800/1006945247 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fc018061ae0 con 0x7fc0000a2a10 2024-03-24T03:11:07.684 INFO:teuthology.orchestra.run.smithi017.stdout:/ 1m 6m3h 2024-03-24T03:11:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.686+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3236352310 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc0000a2a10 msgr2=0x7fc0000a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.686+0000 7fc01e8f1640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc0000a2a10 0x7fc0000a4ed0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc00802efd0 tx=0x7fc00802cf90 comp rx=0 tx=0).stop 2024-03-24T03:11:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.686+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3236352310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0181029a0 msgr2=0x7fc01819e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.686+0000 7fc01e8f1640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0181029a0 0x7fc01819e9b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc01000eb10 tx=0x7fc01000c2d0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.686+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3236352310 shutdown_connections 2024-03-24T03:11:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.686+0000 7fc01e8f1640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fc0000a2a10 0x7fc0000a4ed0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.686+0000 7fc01e8f1640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0181029a0 0x7fc01819e9b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.686+0000 7fc01e8f1640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0181025c0 0x7fc01819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.686+0000 7fc01e8f1640 1 --2- 172.21.15.17:0/3236352310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018100700 0x7fc01819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.686+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3236352310 >> 172.21.15.17:0/3236352310 conn(0x7fc018076af0 msgr2=0x7fc0180fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.686+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3236352310 shutdown_connections 2024-03-24T03:11:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.686+0000 7fc01e8f1640 1 -- 172.21.15.17:0/3236352310 wait complete. 2024-03-24T03:11:07.697 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-24T03:11:07.711 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T03:11:07.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.813+0000 7f2523c81640 1 -- 172.21.15.17:0/2714154244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f251c1016f0 msgr2=0x7f251c10d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.813+0000 7f2523c81640 1 --2- 172.21.15.17:0/2714154244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f251c1016f0 0x7f251c10d430 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2514009cd0 tx=0x7f251402c0f0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.813+0000 7f2523c81640 1 -- 172.21.15.17:0/2714154244 shutdown_connections 2024-03-24T03:11:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.813+0000 7f2523c81640 1 --2- 172.21.15.17:0/2714154244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f251c1016f0 0x7f251c10d430 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.813+0000 7f2523c81640 1 --2- 172.21.15.17:0/2714154244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f251c100d30 0x7f251c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.813+0000 7f2523c81640 1 --2- 172.21.15.17:0/2714154244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f251c1066b0 0x7f251c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.813+0000 7f2523c81640 1 -- 172.21.15.17:0/2714154244 >> 172.21.15.17:0/2714154244 conn(0x7f251c076b30 msgr2=0x7f251c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.813+0000 7f2523c81640 1 -- 172.21.15.17:0/2714154244 shutdown_connections 2024-03-24T03:11:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.813+0000 7f2523c81640 1 -- 172.21.15.17:0/2714154244 wait complete. 2024-03-24T03:11:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f2523c81640 1 Processor -- start 2024-03-24T03:11:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f2523c81640 1 -- start start 2024-03-24T03:11:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f2523c81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f251c100d30 0x7f251c196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f2523c81640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f251c1016f0 0x7f251c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f2523c81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f251c1066b0 0x7f251c19b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f2523c81640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c110980 con 0x7f251c1066b0 2024-03-24T03:11:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f2523c81640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c110800 con 0x7f251c100d30 2024-03-24T03:11:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f2523c81640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c110b00 con 0x7f251c1016f0 2024-03-24T03:11:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f252221e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f251c1066b0 0x7f251c19b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f2521a1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f251c100d30 0x7f251c196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f252221e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f251c1066b0 0x7f251c19b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44068/0 (socket says 172.21.15.17:44068) 2024-03-24T03:11:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f252221e640 1 -- 172.21.15.17:0/3630443511 learned_addr learned my addr 172.21.15.17:0/3630443511 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.814+0000 7f252121c640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f251c1016f0 0x7f251c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.815+0000 7f252221e640 1 -- 172.21.15.17:0/3630443511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f251c1016f0 msgr2=0x7f251c1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.815+0000 7f252221e640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f251c1016f0 0x7f251c1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.815+0000 7f252221e640 1 -- 172.21.15.17:0/3630443511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f251c100d30 msgr2=0x7f251c196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.815+0000 7f252221e640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f251c100d30 0x7f251c196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.815+0000 7f252221e640 1 -- 172.21.15.17:0/3630443511 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c19bd70 con 0x7f251c1066b0 2024-03-24T03:11:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.815+0000 7f2521a1d640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f251c100d30 0x7f251c196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:11:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.815+0000 7f252221e640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f251c1066b0 0x7f251c19b640 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2514004af0 tx=0x7f2514032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.815+0000 7f2512ffd640 1 -- 172.21.15.17:0/3630443511 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514042020 con 0x7f251c1066b0 2024-03-24T03:11:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.815+0000 7f2512ffd640 1 -- 172.21.15.17:0/3630443511 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2514032e80 con 0x7f251c1066b0 2024-03-24T03:11:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.815+0000 7f2523c81640 1 -- 172.21.15.17:0/3630443511 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251c19c000 con 0x7f251c1066b0 2024-03-24T03:11:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.815+0000 7f2512ffd640 1 -- 172.21.15.17:0/3630443511 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514039040 con 0x7f251c1066b0 2024-03-24T03:11:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.815+0000 7f2523c81640 1 -- 172.21.15.17:0/3630443511 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c1a38f0 con 0x7f251c1066b0 2024-03-24T03:11:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.816+0000 7f2523c81640 1 -- 172.21.15.17:0/3630443511 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251c0688e0 con 0x7f251c1066b0 2024-03-24T03:11:07.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.817+0000 7f2512ffd640 1 -- 172.21.15.17:0/3630443511 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2514045020 con 0x7f251c1066b0 2024-03-24T03:11:07.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.818+0000 7f2512ffd640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f25080a2970 0x7f25080a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:07.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.818+0000 7f2521a1d640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f25080a2970 0x7f25080a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:07.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.818+0000 7f2512ffd640 1 -- 172.21.15.17:0/3630443511 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9708+0+0 (secure 0 0 0) 0x7f25140f14a0 con 0x7f251c1066b0 2024-03-24T03:11:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.818+0000 7f2521a1d640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f25080a2970 0x7f25080a4e30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f251c073c00 tx=0x7f2518005f90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:07.821+0000 7f2512ffd640 1 -- 172.21.15.17:0/3630443511 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f251404c020 con 0x7f251c1066b0 2024-03-24T03:11:08.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.006+0000 7f2523c81640 1 -- 172.21.15.17:0/3630443511 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f251c108ad0 con 0x7f251c1066b0 2024-03-24T03:11:08.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.007+0000 7f2512ffd640 1 -- 172.21.15.17:0/3630443511 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+25580 (secure 0 0 0) 0x7f25140bde90 con 0x7f251c1066b0 2024-03-24T03:11:08.008 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:11:08.009 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":101,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T03:10:15.542062+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":13,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":13,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/934316425":"2024-03-25T03:01:04.109518+0000","172.21.15.17:6800/2663792883":"2024-03-25T03:01:04.109518+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/1698088665":"2024-03-25T03:01:04.109518+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/2663792883":"2024-03-25T03:01:04.109518+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/3359268740":"2024-03-25T03:01:04.109518+0000","172.21.15.17:0/1112257866":"2024-03-24T04:05:56.644241+0000","172.21.15.17:0/1127409383":"2024-03-25T03:01:04.109518+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":4,"length":1},{"begin":13,"length":1}]},{"pool":4,"snaps":[{"begin":4,"length":1},{"begin":13,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T03:11:08.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.009+0000 7f2523c81640 1 -- 172.21.15.17:0/3630443511 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f25080a2970 msgr2=0x7f25080a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.009+0000 7f2523c81640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f25080a2970 0x7f25080a4e30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f251c073c00 tx=0x7f2518005f90 comp rx=0 tx=0).stop 2024-03-24T03:11:08.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.009+0000 7f2523c81640 1 -- 172.21.15.17:0/3630443511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f251c1066b0 msgr2=0x7f251c19b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.009+0000 7f2523c81640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f251c1066b0 0x7f251c19b640 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2514004af0 tx=0x7f2514032c90 comp rx=0 tx=0).stop 2024-03-24T03:11:08.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.009+0000 7f2523c81640 1 -- 172.21.15.17:0/3630443511 shutdown_connections 2024-03-24T03:11:08.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.009+0000 7f2523c81640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f25080a2970 0x7f25080a4e30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.009+0000 7f2523c81640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f251c1066b0 0x7f251c19b640 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.009+0000 7f2523c81640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f251c1016f0 0x7f251c1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.009+0000 7f2523c81640 1 --2- 172.21.15.17:0/3630443511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f251c100d30 0x7f251c196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.010+0000 7f2523c81640 1 -- 172.21.15.17:0/3630443511 >> 172.21.15.17:0/3630443511 conn(0x7f251c076b30 msgr2=0x7f251c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:08.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.010+0000 7f2523c81640 1 -- 172.21.15.17:0/3630443511 shutdown_connections 2024-03-24T03:11:08.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.010+0000 7f2523c81640 1 -- 172.21.15.17:0/3630443511 wait complete. 2024-03-24T03:11:08.022 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T02:53:54.207139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-24T02:58:34.785192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-24T02:58:35.679772+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-24T02:58:39.221772+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-24T03:11:08.022 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T03:11:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.130+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/4025297823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0a4106630 msgr2=0x7fd0a4106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.130+0000 7fd0ab7bc640 1 --2- 172.21.15.17:0/4025297823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0a4106630 0x7fd0a4106a10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009900 tx=0x7fd0a002bc30 comp rx=0 tx=0).stop 2024-03-24T03:11:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.130+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/4025297823 shutdown_connections 2024-03-24T03:11:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.130+0000 7fd0ab7bc640 1 --2- 172.21.15.17:0/4025297823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4100f30 0x7fd0a410d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.130+0000 7fd0ab7bc640 1 --2- 172.21.15.17:0/4025297823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a4100570 0x7fd0a41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.130+0000 7fd0ab7bc640 1 --2- 172.21.15.17:0/4025297823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0a4106630 0x7fd0a4106a10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.130+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/4025297823 >> 172.21.15.17:0/4025297823 conn(0x7fd0a4076920 msgr2=0x7fd0a4076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.130+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/4025297823 shutdown_connections 2024-03-24T03:11:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.130+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/4025297823 wait complete. 2024-03-24T03:11:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0ab7bc640 1 Processor -- start 2024-03-24T03:11:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0ab7bc640 1 -- start start 2024-03-24T03:11:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0ab7bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4100570 0x7fd0a4197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:08.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0ab7bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0a4100f30 0x7fd0a4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0ab7bc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a4106630 0x7fd0a419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0ab7bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a4110910 con 0x7fd0a4100f30 2024-03-24T03:11:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0ab7bc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a4110790 con 0x7fd0a4100570 2024-03-24T03:11:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0ab7bc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a4110a90 con 0x7fd0a4106630 2024-03-24T03:11:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0a8d57640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0a4100f30 0x7fd0a4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0a8d57640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0a4100f30 0x7fd0a4198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44098/0 (socket says 172.21.15.17:44098) 2024-03-24T03:11:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0a8d57640 1 -- 172.21.15.17:0/980239404 learned_addr learned my addr 172.21.15.17:0/980239404 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0a9558640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4100570 0x7fd0a4197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0a9d59640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a4106630 0x7fd0a419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0a8d57640 1 -- 172.21.15.17:0/980239404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a4106630 msgr2=0x7fd0a419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0a8d57640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a4106630 0x7fd0a419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0a8d57640 1 -- 172.21.15.17:0/980239404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4100570 msgr2=0x7fd0a4197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0a8d57640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4100570 0x7fd0a4197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.131+0000 7fd0a8d57640 1 -- 172.21.15.17:0/980239404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a419ced0 con 0x7fd0a4100f30 2024-03-24T03:11:08.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.132+0000 7fd0a8d57640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0a4100f30 0x7fd0a4198420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd09400e9f0 tx=0x7fd09400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:08.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.132+0000 7fd09a7fc640 1 -- 172.21.15.17:0/980239404 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09400cd80 con 0x7fd0a4100f30 2024-03-24T03:11:08.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.132+0000 7fd09a7fc640 1 -- 172.21.15.17:0/980239404 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd094005500 con 0x7fd0a4100f30 2024-03-24T03:11:08.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.132+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/980239404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a419d1c0 con 0x7fd0a4100f30 2024-03-24T03:11:08.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.132+0000 7fd09a7fc640 1 -- 172.21.15.17:0/980239404 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094010690 con 0x7fd0a4100f30 2024-03-24T03:11:08.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.132+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/980239404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a41a4a50 con 0x7fd0a4100f30 2024-03-24T03:11:08.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.133+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/980239404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a4101c90 con 0x7fd0a4100f30 2024-03-24T03:11:08.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.134+0000 7fd09a7fc640 1 -- 172.21.15.17:0/980239404 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd0940046c0 con 0x7fd0a4100f30 2024-03-24T03:11:08.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.134+0000 7fd09a7fc640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fd0900a2970 0x7fd0900a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:08.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.134+0000 7fd0a9558640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fd0900a2970 0x7fd0900a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:08.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.134+0000 7fd09a7fc640 1 -- 172.21.15.17:0/980239404 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9796+0+0 (secure 0 0 0) 0x7fd0940cf7b0 con 0x7fd0a4100f30 2024-03-24T03:11:08.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.135+0000 7fd0a9558640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fd0900a2970 0x7fd0900a4e30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009230 tx=0x7fd0a0005fd0 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:08.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.139+0000 7fd09a7fc640 1 -- 172.21.15.17:0/980239404 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd09409bfb0 con 0x7fd0a4100f30 2024-03-24T03:11:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.326+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/980239404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0a4199230 con 0x7fd0a4100f30 2024-03-24T03:11:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.326+0000 7fd09a7fc640 1 -- 172.21.15.17:0/980239404 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd09409b8b0 con 0x7fd0a4100f30 2024-03-24T03:11:08.327 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-24T03:11:08.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.328+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/980239404 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fd0900a2970 msgr2=0x7fd0900a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.328+0000 7fd0ab7bc640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fd0900a2970 0x7fd0900a4e30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009230 tx=0x7fd0a0005fd0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.328+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/980239404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0a4100f30 msgr2=0x7fd0a4198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.328+0000 7fd0ab7bc640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0a4100f30 0x7fd0a4198420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd09400e9f0 tx=0x7fd09400edf0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.328+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/980239404 shutdown_connections 2024-03-24T03:11:08.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.328+0000 7fd0ab7bc640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fd0900a2970 0x7fd0900a4e30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.328+0000 7fd0ab7bc640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a4106630 0x7fd0a419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.328+0000 7fd0ab7bc640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0a4100f30 0x7fd0a4198420 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.328+0000 7fd0ab7bc640 1 --2- 172.21.15.17:0/980239404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4100570 0x7fd0a4197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.328+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/980239404 >> 172.21.15.17:0/980239404 conn(0x7fd0a4076920 msgr2=0x7fd0a40ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:08.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.328+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/980239404 shutdown_connections 2024-03-24T03:11:08.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.328+0000 7fd0ab7bc640 1 -- 172.21.15.17:0/980239404 wait complete. 2024-03-24T03:11:08.338 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:08 smithi017 ceph-mon[31926]: pgmap v311: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-24T03:11:08.338 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:08 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3630443511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:11:08.340 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-24T03:11:08.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.446+0000 7fbe555ff640 1 -- 172.21.15.17:0/943279443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe50107660 msgr2=0x7fbe50107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.446+0000 7fbe555ff640 1 --2- 172.21.15.17:0/943279443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe50107660 0x7fbe50107a40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009930 tx=0x7fbe4002bc30 comp rx=0 tx=0).stop 2024-03-24T03:11:08.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.446+0000 7fbe555ff640 1 -- 172.21.15.17:0/943279443 shutdown_connections 2024-03-24T03:11:08.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.446+0000 7fbe555ff640 1 --2- 172.21.15.17:0/943279443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe50101f80 0x7fbe5010e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.446+0000 7fbe555ff640 1 --2- 172.21.15.17:0/943279443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe501015c0 0x7fbe50101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.446+0000 7fbe555ff640 1 --2- 172.21.15.17:0/943279443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe50107660 0x7fbe50107a40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.446+0000 7fbe555ff640 1 -- 172.21.15.17:0/943279443 >> 172.21.15.17:0/943279443 conn(0x7fbe500fd0d0 msgr2=0x7fbe500ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.446+0000 7fbe555ff640 1 -- 172.21.15.17:0/943279443 shutdown_connections 2024-03-24T03:11:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.446+0000 7fbe555ff640 1 -- 172.21.15.17:0/943279443 wait complete. 2024-03-24T03:11:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe555ff640 1 Processor -- start 2024-03-24T03:11:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe555ff640 1 -- start start 2024-03-24T03:11:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe555ff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe501015c0 0x7fbe50197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:08.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe555ff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe50101f80 0x7fbe50198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:08.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe555ff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe50107660 0x7fbe5019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:08.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe555ff640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe501119a0 con 0x7fbe50101f80 2024-03-24T03:11:08.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe555ff640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50111820 con 0x7fbe501015c0 2024-03-24T03:11:08.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe555ff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50111b20 con 0x7fbe50107660 2024-03-24T03:11:08.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe50101f80 0x7fbe50198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:08.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe50101f80 0x7fbe50198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44112/0 (socket says 172.21.15.17:44112) 2024-03-24T03:11:08.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe47fff640 1 -- 172.21.15.17:0/4014703439 learned_addr learned my addr 172.21.15.17:0/4014703439 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:08.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe4effd640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe501015c0 0x7fbe50197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:08.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe4f7fe640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe50107660 0x7fbe5019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe47fff640 1 -- 172.21.15.17:0/4014703439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe50107660 msgr2=0x7fbe5019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe47fff640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe50107660 0x7fbe5019c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe47fff640 1 -- 172.21.15.17:0/4014703439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe501015c0 msgr2=0x7fbe50197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe47fff640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe501015c0 0x7fbe50197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe47fff640 1 -- 172.21.15.17:0/4014703439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5019cf30 con 0x7fbe50101f80 2024-03-24T03:11:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.447+0000 7fbe47fff640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe50101f80 0x7fbe50198480 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800ea60 tx=0x7fbe3800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.448+0000 7fbe4cff9640 1 -- 172.21.15.17:0/4014703439 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3800cde0 con 0x7fbe50101f80 2024-03-24T03:11:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.448+0000 7fbe4cff9640 1 -- 172.21.15.17:0/4014703439 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe380035f0 con 0x7fbe50101f80 2024-03-24T03:11:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.448+0000 7fbe555ff640 1 -- 172.21.15.17:0/4014703439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5019d220 con 0x7fbe50101f80 2024-03-24T03:11:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.448+0000 7fbe4cff9640 1 -- 172.21.15.17:0/4014703439 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38010670 con 0x7fbe50101f80 2024-03-24T03:11:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.448+0000 7fbe555ff640 1 -- 172.21.15.17:0/4014703439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe501a4ab0 con 0x7fbe50101f80 2024-03-24T03:11:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.449+0000 7fbe555ff640 1 -- 172.21.15.17:0/4014703439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe50068800 con 0x7fbe50101f80 2024-03-24T03:11:08.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.450+0000 7fbe4cff9640 1 -- 172.21.15.17:0/4014703439 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fbe380037a0 con 0x7fbe50101f80 2024-03-24T03:11:08.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.450+0000 7fbe4cff9640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fbe3c0a29f0 0x7fbe3c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:08.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.451+0000 7fbe4effd640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fbe3c0a29f0 0x7fbe3c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:08.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.451+0000 7fbe4cff9640 1 -- 172.21.15.17:0/4014703439 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9796+0+0 (secure 0 0 0) 0x7fbe380cf600 con 0x7fbe50101f80 2024-03-24T03:11:08.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.451+0000 7fbe4effd640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fbe3c0a29f0 0x7fbe3c0a4eb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbe40000c00 tx=0x7fbe40005fd0 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:08.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.455+0000 7fbe4cff9640 1 -- 172.21.15.17:0/4014703439 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbe3809bf90 con 0x7fbe50101f80 2024-03-24T03:11:08.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:08 smithi188 ceph-mon[36206]: pgmap v311: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-24T03:11:08.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:08 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3630443511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:11:08.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:08 smithi019 ceph-mon[35944]: pgmap v311: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-24T03:11:08.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:08 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3630443511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:11:08.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.641+0000 7fbe555ff640 1 -- 172.21.15.17:0/4014703439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbe50109b30 con 0x7fbe50101f80 2024-03-24T03:11:08.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.642+0000 7fbe4cff9640 1 -- 172.21.15.17:0/4014703439 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbe3809b890 con 0x7fbe50101f80 2024-03-24T03:11:08.642 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T03:11:08.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.644+0000 7fbe555ff640 1 -- 172.21.15.17:0/4014703439 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fbe3c0a29f0 msgr2=0x7fbe3c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.644+0000 7fbe555ff640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fbe3c0a29f0 0x7fbe3c0a4eb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbe40000c00 tx=0x7fbe40005fd0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.644+0000 7fbe555ff640 1 -- 172.21.15.17:0/4014703439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe50101f80 msgr2=0x7fbe50198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.644+0000 7fbe555ff640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe50101f80 0x7fbe50198480 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800ea60 tx=0x7fbe3800ef20 comp rx=0 tx=0).stop 2024-03-24T03:11:08.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.644+0000 7fbe555ff640 1 -- 172.21.15.17:0/4014703439 shutdown_connections 2024-03-24T03:11:08.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.644+0000 7fbe555ff640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fbe3c0a29f0 0x7fbe3c0a4eb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.644+0000 7fbe555ff640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe50107660 0x7fbe5019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.644+0000 7fbe555ff640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe50101f80 0x7fbe50198480 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.644+0000 7fbe555ff640 1 --2- 172.21.15.17:0/4014703439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe501015c0 0x7fbe50197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.644+0000 7fbe555ff640 1 -- 172.21.15.17:0/4014703439 >> 172.21.15.17:0/4014703439 conn(0x7fbe500fd0d0 msgr2=0x7fbe500fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:08.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.644+0000 7fbe555ff640 1 -- 172.21.15.17:0/4014703439 shutdown_connections 2024-03-24T03:11:08.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.644+0000 7fbe555ff640 1 -- 172.21.15.17:0/4014703439 wait complete. 2024-03-24T03:11:08.655 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-24T03:11:08.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.763+0000 7fb4cd190640 1 -- 172.21.15.17:0/637235350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4c8100530 msgr2=0x7fb4c81009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.763+0000 7fb4cd190640 1 --2- 172.21.15.17:0/637235350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4c8100530 0x7fb4c81009b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009900 tx=0x7fb4bc02bc30 comp rx=0 tx=0).stop 2024-03-24T03:11:08.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.763+0000 7fb4cd190640 1 -- 172.21.15.17:0/637235350 shutdown_connections 2024-03-24T03:11:08.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.763+0000 7fb4cd190640 1 --2- 172.21.15.17:0/637235350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8100ef0 0x7fb4c810d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.763+0000 7fb4cd190640 1 --2- 172.21.15.17:0/637235350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4c8100530 0x7fb4c81009b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.763+0000 7fb4cd190640 1 --2- 172.21.15.17:0/637235350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4c81065f0 0x7fb4c81069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.763+0000 7fb4cd190640 1 -- 172.21.15.17:0/637235350 >> 172.21.15.17:0/637235350 conn(0x7fb4c80776a0 msgr2=0x7fb4c8077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:08.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.764+0000 7fb4cd190640 1 -- 172.21.15.17:0/637235350 shutdown_connections 2024-03-24T03:11:08.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.764+0000 7fb4cd190640 1 -- 172.21.15.17:0/637235350 wait complete. 2024-03-24T03:11:08.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.764+0000 7fb4cd190640 1 Processor -- start 2024-03-24T03:11:08.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.764+0000 7fb4cd190640 1 -- start start 2024-03-24T03:11:08.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4cd190640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4c8100530 0x7fb4c8196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:08.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4cd190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8100ef0 0x7fb4c81972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:08.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4cd190640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4c81065f0 0x7fb4c819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:08.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4cd190640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c81108b0 con 0x7fb4c81065f0 2024-03-24T03:11:08.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4cd190640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c8110730 con 0x7fb4c8100ef0 2024-03-24T03:11:08.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4cd190640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c8110a30 con 0x7fb4c8100530 2024-03-24T03:11:08.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4c759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4c81065f0 0x7fb4c819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:08.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4c759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4c81065f0 0x7fb4c819b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44124/0 (socket says 172.21.15.17:44124) 2024-03-24T03:11:08.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4c759e640 1 -- 172.21.15.17:0/1445695384 learned_addr learned my addr 172.21.15.17:0/1445695384 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:08.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4c659c640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8100ef0 0x7fb4c81972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:08.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4c6d9d640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4c8100530 0x7fb4c8196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:08.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4c759e640 1 -- 172.21.15.17:0/1445695384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4c8100530 msgr2=0x7fb4c8196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4c759e640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4c8100530 0x7fb4c8196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4c759e640 1 -- 172.21.15.17:0/1445695384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8100ef0 msgr2=0x7fb4c81972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4c759e640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8100ef0 0x7fb4c81972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4c759e640 1 -- 172.21.15.17:0/1445695384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c819bda0 con 0x7fb4c81065f0 2024-03-24T03:11:08.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4c759e640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4c81065f0 0x7fb4c819b670 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800eb10 tx=0x7fb4b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:08.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4affff640 1 -- 172.21.15.17:0/1445695384 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8004b20 con 0x7fb4c81065f0 2024-03-24T03:11:08.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4cd190640 1 -- 172.21.15.17:0/1445695384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c819c090 con 0x7fb4c81065f0 2024-03-24T03:11:08.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4affff640 1 -- 172.21.15.17:0/1445695384 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4b8004cc0 con 0x7fb4c81065f0 2024-03-24T03:11:08.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4affff640 1 -- 172.21.15.17:0/1445695384 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8010760 con 0x7fb4c81065f0 2024-03-24T03:11:08.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.765+0000 7fb4cd190640 1 -- 172.21.15.17:0/1445695384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c81a3920 con 0x7fb4c81065f0 2024-03-24T03:11:08.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.766+0000 7fb4cd190640 1 -- 172.21.15.17:0/1445695384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c8068490 con 0x7fb4c81065f0 2024-03-24T03:11:08.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.767+0000 7fb4affff640 1 -- 172.21.15.17:0/1445695384 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb4b8005020 con 0x7fb4c81065f0 2024-03-24T03:11:08.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.768+0000 7fb4affff640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb4b00a2970 0x7fb4b00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:08.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.768+0000 7fb4c6d9d640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb4b00a2970 0x7fb4b00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:08.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.768+0000 7fb4affff640 1 -- 172.21.15.17:0/1445695384 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9796+0+0 (secure 0 0 0) 0x7fb4b80cf8e0 con 0x7fb4c81065f0 2024-03-24T03:11:08.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.768+0000 7fb4c6d9d640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb4b00a2970 0x7fb4b00a4e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009bd0 tx=0x7fb4c0005f90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:08.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.772+0000 7fb4affff640 1 -- 172.21.15.17:0/1445695384 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb4b809c140 con 0x7fb4c81065f0 2024-03-24T03:11:08.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.959+0000 7fb4cd190640 1 -- 172.21.15.17:0/1445695384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb4c802d070 con 0x7fb4c81065f0 2024-03-24T03:11:08.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.960+0000 7fb4affff640 1 -- 172.21.15.17:0/1445695384 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb4b809ba40 con 0x7fb4c81065f0 2024-03-24T03:11:08.960 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-24T03:11:08.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.962+0000 7fb4cd190640 1 -- 172.21.15.17:0/1445695384 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb4b00a2970 msgr2=0x7fb4b00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.962+0000 7fb4cd190640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb4b00a2970 0x7fb4b00a4e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009bd0 tx=0x7fb4c0005f90 comp rx=0 tx=0).stop 2024-03-24T03:11:08.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.962+0000 7fb4cd190640 1 -- 172.21.15.17:0/1445695384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4c81065f0 msgr2=0x7fb4c819b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:08.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.962+0000 7fb4cd190640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4c81065f0 0x7fb4c819b670 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800eb10 tx=0x7fb4b800c2d0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.962+0000 7fb4cd190640 1 -- 172.21.15.17:0/1445695384 shutdown_connections 2024-03-24T03:11:08.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.962+0000 7fb4cd190640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb4b00a2970 0x7fb4b00a4e30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.962+0000 7fb4cd190640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4c81065f0 0x7fb4c819b670 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.962+0000 7fb4cd190640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8100ef0 0x7fb4c81972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.962+0000 7fb4cd190640 1 --2- 172.21.15.17:0/1445695384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4c8100530 0x7fb4c8196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:08.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.962+0000 7fb4cd190640 1 -- 172.21.15.17:0/1445695384 >> 172.21.15.17:0/1445695384 conn(0x7fb4c80776a0 msgr2=0x7fb4c80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:08.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.962+0000 7fb4cd190640 1 -- 172.21.15.17:0/1445695384 shutdown_connections 2024-03-24T03:11:08.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:08.962+0000 7fb4cd190640 1 -- 172.21.15.17:0/1445695384 wait complete. 2024-03-24T03:11:08.973 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-24T03:11:09.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.082+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3597773169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c810bb40 msgr2=0x7fd0c810df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:09.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.082+0000 7fd0ccf9d640 1 --2- 172.21.15.17:0/3597773169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c810bb40 0x7fd0c810df90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009ca0 tx=0x7fd0c002c180 comp rx=0 tx=0).stop 2024-03-24T03:11:09.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.082+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3597773169 shutdown_connections 2024-03-24T03:11:09.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.082+0000 7fd0ccf9d640 1 --2- 172.21.15.17:0/3597773169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c810bb40 0x7fd0c810df90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.082+0000 7fd0ccf9d640 1 --2- 172.21.15.17:0/3597773169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0c8103090 0x7fd0c810b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.082+0000 7fd0ccf9d640 1 --2- 172.21.15.17:0/3597773169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c8102770 0x7fd0c8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.082+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3597773169 >> 172.21.15.17:0/3597773169 conn(0x7fd0c80fbef0 msgr2=0x7fd0c80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:09.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.082+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3597773169 shutdown_connections 2024-03-24T03:11:09.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.082+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3597773169 wait complete. 2024-03-24T03:11:09.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.082+0000 7fd0ccf9d640 1 Processor -- start 2024-03-24T03:11:09.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.082+0000 7fd0ccf9d640 1 -- start start 2024-03-24T03:11:09.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0ccf9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c8102770 0x7fd0c8197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:09.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0ccf9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c8103090 0x7fd0c8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:09.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0ccf9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0c810bb40 0x7fd0c819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:09.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0ccf9d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c81118c0 con 0x7fd0c8102770 2024-03-24T03:11:09.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0ccf9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c8111740 con 0x7fd0c8103090 2024-03-24T03:11:09.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0ccf9d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c8111a40 con 0x7fd0c810bb40 2024-03-24T03:11:09.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0c77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c8102770 0x7fd0c8197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:09.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0c77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c8102770 0x7fd0c8197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44144/0 (socket says 172.21.15.17:44144) 2024-03-24T03:11:09.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0c77fe640 1 -- 172.21.15.17:0/3177474508 learned_addr learned my addr 172.21.15.17:0/3177474508 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:09.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0c7fff640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0c810bb40 0x7fd0c819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:09.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0c6ffd640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c8103090 0x7fd0c8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:09.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0c77fe640 1 -- 172.21.15.17:0/3177474508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0c810bb40 msgr2=0x7fd0c819c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:09.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0c77fe640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0c810bb40 0x7fd0c819c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0c77fe640 1 -- 172.21.15.17:0/3177474508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c8103090 msgr2=0x7fd0c8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:09.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0c77fe640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c8103090 0x7fd0c8198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0c77fe640 1 -- 172.21.15.17:0/3177474508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c819cd80 con 0x7fd0c8102770 2024-03-24T03:11:09.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0c77fe640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c8102770 0x7fd0c8197de0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800ea60 tx=0x7fd0b800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:09.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0c4ff9640 1 -- 172.21.15.17:0/3177474508 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b800cde0 con 0x7fd0c8102770 2024-03-24T03:11:09.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.083+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3177474508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c819d070 con 0x7fd0c8102770 2024-03-24T03:11:09.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.084+0000 7fd0c4ff9640 1 -- 172.21.15.17:0/3177474508 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0b80035f0 con 0x7fd0c8102770 2024-03-24T03:11:09.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.084+0000 7fd0c4ff9640 1 -- 172.21.15.17:0/3177474508 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8010670 con 0x7fd0c8102770 2024-03-24T03:11:09.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.084+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3177474508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c81a4900 con 0x7fd0c8102770 2024-03-24T03:11:09.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.085+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3177474508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c80683b0 con 0x7fd0c8102770 2024-03-24T03:11:09.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.085+0000 7fd0c4ff9640 1 -- 172.21.15.17:0/3177474508 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd0b80037a0 con 0x7fd0c8102770 2024-03-24T03:11:09.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.086+0000 7fd0c4ff9640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fd0b00a2970 0x7fd0b00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:09.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.086+0000 7fd0c6ffd640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fd0b00a2970 0x7fd0b00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:09.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.086+0000 7fd0c4ff9640 1 -- 172.21.15.17:0/3177474508 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9796+0+0 (secure 0 0 0) 0x7fd0b80cf650 con 0x7fd0c8102770 2024-03-24T03:11:09.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.087+0000 7fd0c6ffd640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fd0b00a2970 0x7fd0b00a4e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd0c810cdf0 tx=0x7fd0bc005f90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:09.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.090+0000 7fd0c4ff9640 1 -- 172.21.15.17:0/3177474508 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd0b809beb0 con 0x7fd0c8102770 2024-03-24T03:11:09.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.273+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3177474508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd0c8106c30 con 0x7fd0c8102770 2024-03-24T03:11:09.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.274+0000 7fd0c4ff9640 1 -- 172.21.15.17:0/3177474508 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v102) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd0b809b7b0 con 0x7fd0c8102770 2024-03-24T03:11:09.274 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T03:11:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.275+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3177474508 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fd0b00a2970 msgr2=0x7fd0b00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.275+0000 7fd0ccf9d640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fd0b00a2970 0x7fd0b00a4e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd0c810cdf0 tx=0x7fd0bc005f90 comp rx=0 tx=0).stop 2024-03-24T03:11:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.275+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3177474508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c8102770 msgr2=0x7fd0c8197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.275+0000 7fd0ccf9d640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c8102770 0x7fd0c8197de0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800ea60 tx=0x7fd0b800ef20 comp rx=0 tx=0).stop 2024-03-24T03:11:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.275+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3177474508 shutdown_connections 2024-03-24T03:11:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.275+0000 7fd0ccf9d640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fd0b00a2970 0x7fd0b00a4e30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.275+0000 7fd0ccf9d640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0c810bb40 0x7fd0c819c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.275+0000 7fd0ccf9d640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c8103090 0x7fd0c8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.275+0000 7fd0ccf9d640 1 --2- 172.21.15.17:0/3177474508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c8102770 0x7fd0c8197de0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.275+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3177474508 >> 172.21.15.17:0/3177474508 conn(0x7fd0c80fbef0 msgr2=0x7fd0c80fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:09.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.276+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3177474508 shutdown_connections 2024-03-24T03:11:09.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.276+0000 7fd0ccf9d640 1 -- 172.21.15.17:0/3177474508 wait complete. 2024-03-24T03:11:09.286 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:09 smithi017 ceph-mon[31926]: from='client.25430 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:11:09.286 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:09 smithi017 ceph-mon[31926]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:11:09.286 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:09 smithi017 ceph-mon[31926]: osdmap e102: 12 total, 12 up, 12 in 2024-03-24T03:11:09.286 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:09 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/980239404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:11:09.286 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:09 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/4014703439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:11:09.286 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:09 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1445695384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:11:09.287 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.389+0000 7f6c7ca61640 1 -- 172.21.15.17:0/1312631084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c781028d0 msgr2=0x7f6c78102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.389+0000 7f6c7ca61640 1 --2- 172.21.15.17:0/1312631084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c781028d0 0x7f6c78102cb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009900 tx=0x7f6c6802bcd0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.389+0000 7f6c7ca61640 1 -- 172.21.15.17:0/1312631084 shutdown_connections 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.389+0000 7f6c7ca61640 1 --2- 172.21.15.17:0/1312631084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c7810bca0 0x7f6c7810e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.389+0000 7f6c7ca61640 1 --2- 172.21.15.17:0/1312631084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c781031f0 0x7f6c7810b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.389+0000 7f6c7ca61640 1 --2- 172.21.15.17:0/1312631084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c781028d0 0x7f6c78102cb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.389+0000 7f6c7ca61640 1 -- 172.21.15.17:0/1312631084 >> 172.21.15.17:0/1312631084 conn(0x7f6c78076920 msgr2=0x7f6c78076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.389+0000 7f6c7ca61640 1 -- 172.21.15.17:0/1312631084 shutdown_connections 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.389+0000 7f6c7ca61640 1 -- 172.21.15.17:0/1312631084 wait complete. 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.390+0000 7f6c7ca61640 1 Processor -- start 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.390+0000 7f6c7ca61640 1 -- start start 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.390+0000 7f6c7ca61640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c781028d0 0x7f6c78197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:09.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.390+0000 7f6c7ca61640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c781031f0 0x7f6c78198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.390+0000 7f6c7ca61640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7810bca0 0x7f6c7819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.390+0000 7f6c7ca61640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78111a30 con 0x7f6c7810bca0 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.390+0000 7f6c7ca61640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c781118b0 con 0x7f6c781028d0 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.390+0000 7f6c7ca61640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78111bb0 con 0x7f6c781031f0 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.390+0000 7f6c76d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7810bca0 0x7f6c7819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.390+0000 7f6c76d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7810bca0 0x7f6c7819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44164/0 (socket says 172.21.15.17:44164) 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c76d9d640 1 -- 172.21.15.17:0/796736014 learned_addr learned my addr 172.21.15.17:0/796736014 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c6ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c781031f0 0x7f6c78198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c7659c640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c781028d0 0x7f6c78197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c76d9d640 1 -- 172.21.15.17:0/796736014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c781031f0 msgr2=0x7f6c78198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c76d9d640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c781031f0 0x7f6c78198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c76d9d640 1 -- 172.21.15.17:0/796736014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c781028d0 msgr2=0x7f6c78197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c76d9d640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c781028d0 0x7f6c78197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:09.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c76d9d640 1 -- 172.21.15.17:0/796736014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7819cf20 con 0x7f6c7810bca0 2024-03-24T03:11:09.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c76d9d640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7810bca0 0x7f6c7819c7f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000eb40 tx=0x7f6c7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:09.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c6f7fe640 1 -- 172.21.15.17:0/796736014 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70004b20 con 0x7f6c7810bca0 2024-03-24T03:11:09.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c6f7fe640 1 -- 172.21.15.17:0/796736014 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c70004cc0 con 0x7f6c7810bca0 2024-03-24T03:11:09.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c7ca61640 1 -- 172.21.15.17:0/796736014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7819d210 con 0x7f6c7810bca0 2024-03-24T03:11:09.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c6f7fe640 1 -- 172.21.15.17:0/796736014 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70010850 con 0x7f6c7810bca0 2024-03-24T03:11:09.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.391+0000 7f6c7ca61640 1 -- 172.21.15.17:0/796736014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c781a4aa0 con 0x7f6c7810bca0 2024-03-24T03:11:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.392+0000 7f6c7ca61640 1 -- 172.21.15.17:0/796736014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7806a4f0 con 0x7f6c7810bca0 2024-03-24T03:11:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.393+0000 7f6c6f7fe640 1 -- 172.21.15.17:0/796736014 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6c70005020 con 0x7f6c7810bca0 2024-03-24T03:11:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.394+0000 7f6c6f7fe640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f6c640a29f0 0x7f6c640a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.394+0000 7f6c7659c640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f6c640a29f0 0x7f6c640a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.394+0000 7f6c6f7fe640 1 -- 172.21.15.17:0/796736014 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9796+0+0 (secure 0 0 0) 0x7f6c700cfa40 con 0x7f6c7810bca0 2024-03-24T03:11:09.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.394+0000 7f6c7659c640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f6c640a29f0 0x7f6c640a4eb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6c6802efd0 tx=0x7f6c6802cf90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:09.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.398+0000 7f6c6f7fe640 1 -- 172.21.15.17:0/796736014 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6c7009c2a0 con 0x7f6c7810bca0 2024-03-24T03:11:09.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:09 smithi188 ceph-mon[36206]: from='client.25430 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:11:09.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:09 smithi188 ceph-mon[36206]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:11:09.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:09 smithi188 ceph-mon[36206]: osdmap e102: 12 total, 12 up, 12 in 2024-03-24T03:11:09.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:09 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/980239404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:11:09.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:09 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/4014703439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:11:09.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:09 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1445695384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:11:09.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:09 smithi019 ceph-mon[35944]: from='client.25430 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:11:09.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:09 smithi019 ceph-mon[35944]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T03:11:09.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:09 smithi019 ceph-mon[35944]: osdmap e102: 12 total, 12 up, 12 in 2024-03-24T03:11:09.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:09 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/980239404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:11:09.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:09 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/4014703439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:11:09.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:09 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1445695384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:11:09.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.630+0000 7f6c7ca61640 1 -- 172.21.15.17:0/796736014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f6c78199280 con 0x7f6c7810bca0 2024-03-24T03:11:09.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.639+0000 7f6c6f7fe640 1 -- 172.21.15.17:0/796736014 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+740361 (secure 0 0 0) 0x7f6c7009bba0 con 0x7f6c7810bca0 2024-03-24T03:11:09.649 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:11:09.649 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:11:09.649 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 25, 2024-03-24T03:11:09.649 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.649 INFO:teuthology.orchestra.run.smithi017.stdout: "active_gid": 25334, 2024-03-24T03:11:09.649 INFO:teuthology.orchestra.run.smithi017.stdout: "active_name": "x", 2024-03-24T03:11:09.649 INFO:teuthology.orchestra.run.smithi017.stdout: "active_addrs": { 2024-03-24T03:11:09.649 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-03-24T03:11:09.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.649 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1006945247 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1006945247 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: "active_addr": "172.21.15.17:6801/1006945247", 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: "active_change": "2024-03-24T03:01:04.109773+0000", 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: "active_mgr_features": 4540701547738038271, 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: "available": true, 2024-03-24T03:11:09.650 INFO:teuthology.orchestra.run.smithi017.stdout: "standbys": [ 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "gid": 15567, 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "y", 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_features": 4540701547738038271, 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "available_modules": [ 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "alerts", 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How frequently to reexamine health status", 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_destination": { 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_destination", 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Email address to send alerts to", 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_from_name": { 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_from_name", 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "Ceph", 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Email From: name", 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_host": { 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_host", 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP server", 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_password": { 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_password", 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Password to authenticate with", 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_port": { 2024-03-24T03:11:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_port", 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "465", 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP port", 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_sender": { 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_sender", 2024-03-24T03:11:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP envelope sender", 2024-03-24T03:11:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_ssl": { 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_ssl", 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_user": { 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_user", 2024-03-24T03:11:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "User to authenticate as", 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.664 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "balancer", 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "active": { 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "active", 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "automatically balance PGs across cluster", 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "begin_time": { 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "begin_time", 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0000", 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "beginning time of day to automatically balance", 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.666 INFO:teuthology.orchestra.run.smithi017.stdout: "begin_weekday": { 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "begin_weekday", 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0", 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "6", 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_max_iterations": { 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_max_iterations", 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:09.667 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "25", 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "250", 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_metrics": { 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_metrics", 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.668 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "pgs,objects,bytes", 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_step": { 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_step", 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0.5", 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0.001", 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "0.999", 2024-03-24T03:11:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "aggressiveness of optimization", 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "end_time": { 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "end_time", 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "2359", 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.670 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "ending time of day to automatically balance", 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "end_weekday": { 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "end_weekday", 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0", 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "6", 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.671 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.672 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.673 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.674 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.675 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "min_score": { 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "min_score", 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.676 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": { 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mode", 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "upmap", 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-compat", 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "none", 2024-03-24T03:11:09.677 INFO:teuthology.orchestra.run.smithi017.stdout: "read", 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap", 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap-read" 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Balancer mode", 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_ids": { 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pool_ids", 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.678 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-24T03:11:09.679 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap_max_deviation": { 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "upmap_max_deviation", 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.680 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap_max_optimizations": { 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "upmap_max_optimizations", 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephadm", 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_down_multiplier": { 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_down_multiplier", 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3.0", 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-24T03:11:09.682 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_refresh_rate": { 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_refresh_rate", 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "20", 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.683 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_starting_port": { 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_starting_port", 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "4721", 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.684 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_ptrace": { 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_ptrace", 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.685 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "autotune_interval": { 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "autotune_interval", 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.686 INFO:teuthology.orchestra.run.smithi017.stdout: "autotune_memory_target_ratio": { 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "autotune_memory_target_ratio", 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0.7", 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm_log_destination": { 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephadm_log_destination", 2024-03-24T03:11:09.687 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "file", 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "file,syslog", 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "syslog" 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.688 INFO:teuthology.orchestra.run.smithi017.stdout: "cgroups_split": { 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cgroups_split", 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "config_checks_enabled": { 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_checks_enabled", 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.689 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "config_dashboard": { 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_dashboard", 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.690 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_alertmanager": { 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_alertmanager", 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.691 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_base": { 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_base", 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.692 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Container image name, without the tag", 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_elasticsearch": { 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_elasticsearch", 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "elasticsearch container image", 2024-03-24T03:11:09.693 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_grafana": { 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_grafana", 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.694 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_haproxy": { 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_haproxy", 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "HAproxy container image", 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.695 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_agent": { 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_agent", 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger agent container image", 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_collector": { 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_collector", 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger collector container image", 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_query": { 2024-03-24T03:11:09.697 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_query", 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger query container image", 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_keepalived": { 2024-03-24T03:11:09.698 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_keepalived", 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Keepalived container image", 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_loki": { 2024-03-24T03:11:09.699 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_loki", 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Loki container image", 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_node_exporter": { 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_node_exporter", 2024-03-24T03:11:09.700 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_nvmeof": { 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_nvmeof", 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.701 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Nvme-of container image", 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_prometheus": { 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_prometheus", 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.702 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_promtail": { 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_promtail", 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.703 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Promtail container image", 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_snmp_gateway": { 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_snmp_gateway", 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-24T03:11:09.704 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SNMP Gateway container image", 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "container_init": { 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_init", 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.705 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Run podman/docker with `--init`", 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: "daemon_cache_timeout": { 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "daemon_cache_timeout", 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:09.706 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "default_cephadm_command_timeout": { 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "default_cephadm_command_timeout", 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "900", 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.707 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "default_registry": { 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "default_registry", 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io", 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.708 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "device_cache_timeout": { 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_cache_timeout", 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1800", 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache device inventory", 2024-03-24T03:11:09.709 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "device_enhanced_scan": { 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_enhanced_scan", 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-24T03:11:09.710 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "facts_cache_timeout": { 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "facts_cache_timeout", 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache host facts data", 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.711 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "host_check_interval": { 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "host_check_interval", 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to perform a host check", 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.712 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "hw_monitoring": { 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "hw_monitoring", 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.713 INFO:teuthology.orchestra.run.smithi017.stdout: "inventory_list_all": { 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "inventory_list_all", 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.714 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "log_refresh_metadata": { 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_refresh_metadata", 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "*", 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "max_count_per_host": { 2024-03-24T03:11:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_count_per_host", 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "max number of daemons per service per host", 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd_draining_count": { 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_osd_draining_count", 2024-03-24T03:11:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "migration_current": { 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "migration_current", 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "internal - do not modify", 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": { 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mode", 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "root", 2024-03-24T03:11:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm-package", 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "root" 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "mode for remote execution of cephadm", 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "oob_default_addr": { 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "oob_default_addr", 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "169.254.1.1", 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "prometheus_alerts_path": { 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prometheus_alerts_path", 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_insecure": { 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_insecure", 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-24T03:11:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_password": { 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_password", 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_url": { 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_url", 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_username": { 2024-03-24T03:11:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_username", 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "secure_monitoring_stack": { 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "secure_monitoring_stack", 2024-03-24T03:11:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "service_discovery_port": { 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "service_discovery_port", 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.731 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8765", 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "cephadm service discovery port", 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "ssh_config_file": { 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssh_config_file", 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.732 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "use_agent": { 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "use_agent", 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-24T03:11:09.733 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "use_repo_digest": { 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "use_repo_digest", 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.734 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_failed_host_check": { 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_failed_host_check", 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if the host check fails", 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_stray_daemons": { 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_stray_daemons", 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_stray_hosts": { 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_stray_hosts", 2024-03-24T03:11:09.736 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crash", 2024-03-24T03:11:09.737 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.738 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.739 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.740 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.741 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "retain_interval": { 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "retain_interval", 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "31536000", 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.742 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_recent_interval": { 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_recent_interval", 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1209600", 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-24T03:11:09.743 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "dashboard", 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:09.744 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "ALERTMANAGER_API_HOST": { 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.745 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.746 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "AUDIT_API_ENABLED": { 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "AUDIT_API_ENABLED", 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.747 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "ENABLE_BROWSABLE_API": { 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-24T03:11:09.749 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.750 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.751 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.752 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.753 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.753 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.753 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.753 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.753 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.753 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-24T03:11:09.753 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-24T03:11:09.753 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.753 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.753 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.754 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.754 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.754 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.754 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.754 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.754 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.754 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.754 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.754 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-24T03:11:09.754 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.755 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.756 INFO:teuthology.orchestra.run.smithi017.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.757 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_PASSWORD": { 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.758 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.759 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_URL": { 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_URL", 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.760 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_USERNAME": { 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.761 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.762 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-24T03:11:09.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-24T03:11:09.764 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "MULTICLUSTER_CONFIG": { 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.765 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "{}", 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "PROMETHEUS_API_HOST": { 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.766 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.767 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.768 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.769 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.770 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.771 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.772 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_ENABLED": { 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-24T03:11:09.775 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.776 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.777 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8", 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "45", 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.778 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_ACCESS_KEY": { 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.779 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.780 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_SECRET_KEY": { 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_SSL_VERIFY": { 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.782 INFO:teuthology.orchestra.run.smithi017.stdout: "UNSAFE_TLS_v1_2": { 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-24T03:11:09.783 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-24T03:11:09.784 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.785 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "cross_origin_url": { 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cross_origin_url", 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.786 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "crt_file": { 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crt_file", 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.787 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "debug": { 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "debug", 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.788 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable/disable debug options", 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "jwt_token_ttl": { 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "jwt_token_ttl", 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "28800", 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.789 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "key_file": { 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "key_file", 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.790 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.791 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.792 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.793 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.794 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "motd": { 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "motd", 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.795 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "The message of the day", 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "redirect_resolve_ip_addr": { 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "redirect_resolve_ip_addr", 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.796 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "::", 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.797 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8080", 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.798 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl": { 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl", 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.799 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl_server_port": { 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl_server_port", 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8443", 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.800 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_behaviour": { 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_behaviour", 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "redirect", 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.801 INFO:teuthology.orchestra.run.smithi017.stdout: "redirect" 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_error_status_code": { 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_error_status_code", 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "500", 2024-03-24T03:11:09.802 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "400", 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "599", 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "url_prefix": { 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "url_prefix", 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.803 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "devicehealth", 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_monitoring": { 2024-03-24T03:11:09.804 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enable_monitoring", 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "monitor device health metrics", 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.805 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.806 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.807 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.808 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.809 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "mark_out_threshold": { 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mark_out_threshold", 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "2419200", 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.810 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": { 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pool_name", 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "device_health_metrics", 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.811 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "retention_period": { 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "retention_period", 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15552000", 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long to retain device health metrics", 2024-03-24T03:11:09.812 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "scrape_frequency": { 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "scrape_frequency", 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "86400", 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.813 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "self_heal": { 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "self_heal", 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to wake up and check device health", 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.815 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_threshold": { 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_threshold", 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "7257600", 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "diskprediction_local", 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.817 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.818 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.819 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.820 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "predict_interval": { 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "predict_interval", 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.821 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "86400", 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "predictor_model": { 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "predictor_model", 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.822 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "prophetstor", 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.823 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "influx", 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": false, 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "influxdb python module not found", 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.824 INFO:teuthology.orchestra.run.smithi017.stdout: "batch_size": { 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "batch_size", 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5000", 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "database": { 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "database", 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.825 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "ceph", 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": { 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "hostname", 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.826 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB server hostname", 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "30", 2024-03-24T03:11:09.827 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "5", 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.828 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.829 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.829 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.829 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.829 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.829 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.829 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.829 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.829 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.829 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.829 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.829 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.829 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.830 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.830 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.830 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.830 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.830 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.830 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.830 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.830 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.830 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.830 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.830 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.831 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.832 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: "password": { 2024-03-24T03:11:09.833 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "password", 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "password of InfluxDB server user", 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "port": { 2024-03-24T03:11:09.834 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "port", 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8086", 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB server port", 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl": { 2024-03-24T03:11:09.835 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl", 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "false", 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.836 INFO:teuthology.orchestra.run.smithi017.stdout: "threads": { 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "threads", 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "32", 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "username": { 2024-03-24T03:11:09.837 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "username", 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "username of InfluxDB server user", 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "verify_ssl": { 2024-03-24T03:11:09.838 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "verify_ssl", 2024-03-24T03:11:09.839 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.839 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.839 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.839 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "true", 2024-03-24T03:11:09.839 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.839 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.839 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.839 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-24T03:11:09.839 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.839 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.839 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.839 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "insights", 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.840 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.841 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.843 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.844 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "iostat", 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.846 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.847 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.849 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.850 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "k8sevents", 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.851 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_event_retention_days": { 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ceph_event_retention_days", 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "7", 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "config_check_secs": { 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_check_secs", 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "10", 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.854 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.855 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.856 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.856 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.856 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.856 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.856 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.856 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.856 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.856 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.856 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.857 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.857 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.857 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.857 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.857 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.857 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.857 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.857 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.857 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.858 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.858 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.858 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.858 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.858 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.858 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.858 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.858 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.858 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.859 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "localpool", 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: "failure_domain": { 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "failure_domain", 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.860 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "host", 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "failure domain for any created local pool", 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.861 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.862 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.863 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.864 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.865 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": { 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "min_size", 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.866 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default min_size for any created local pool", 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "num_rep": { 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "num_rep", 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.867 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3", 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default replica count for any created local pool", 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": { 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pg_num", 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.868 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "128", 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default pg_num for any created local pool", 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: "prefix": { 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prefix", 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.869 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "name prefix for any created local pool", 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "subtree": { 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "subtree", 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.870 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "rack", 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mds_autoscaler", 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.871 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.872 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.873 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.874 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.875 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mirroring", 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.876 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.877 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.878 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.879 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.880 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "nfs", 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.882 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.883 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.884 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.885 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "orchestrator", 2024-03-24T03:11:09.886 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "fail_fs": { 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "fail_fs", 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.888 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.891 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator": { 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "orchestrator", 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.892 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm", 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: "rook", 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: "test_orchestrator" 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Orchestrator backend", 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "osd_perf_query", 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.893 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.894 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.895 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.896 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.897 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "osd_support", 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.898 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.899 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.900 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.901 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.902 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pg_autoscaler", 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.903 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.904 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.905 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.906 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.907 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.908 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "threshold": { 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "threshold", 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3.0", 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1.0", 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.909 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "scaling threshold", 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "progress", 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_pg_recovery_event": { 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_pg_recovery_event", 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.910 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "enabled": { 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enabled", 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.911 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.912 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.913 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.914 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.915 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.916 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "max_completed_events": { 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_completed_events", 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "50", 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "number of past completed events to remember", 2024-03-24T03:11:09.917 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.918 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long the module is going to sleep", 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prometheus", 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: "cache": { 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cache", 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.919 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "exclude_perf_counters": { 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "exclude_perf_counters", 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.920 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.921 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.922 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.923 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.924 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.925 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_stats_pools": { 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_stats_pools", 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.926 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "300", 2024-03-24T03:11:09.927 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "scrape_interval": { 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "scrape_interval", 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15.0", 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.928 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "::", 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.929 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "9283", 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-24T03:11:09.930 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "stale_cache_strategy": { 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "stale_cache_strategy", 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "log", 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.931 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_behaviour": { 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_behaviour", 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "default", 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "default", 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "error" 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.932 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_error_status_code": { 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_error_status_code", 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "500", 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "400", 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "599", 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.933 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_support", 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.934 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.935 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.937 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.938 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "max_concurrent_snap_create": { 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_concurrent_snap_create", 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.939 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "mirror_snapshot_schedule": { 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mirror_snapshot_schedule", 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.940 INFO:teuthology.orchestra.run.smithi017.stdout: "trash_purge_schedule": { 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "trash_purge_schedule", 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.941 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "restful", 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_auth": { 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enable_auth", 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.942 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "key_file": { 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "key_file", 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.943 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.944 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.944 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.944 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.944 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.944 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.944 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.944 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.944 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.944 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.945 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.946 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.947 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.948 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.949 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rgw", 2024-03-24T03:11:09.950 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.951 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.952 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.953 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.954 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rook", 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.955 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.956 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.957 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.958 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "storage_class": { 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "storage_class", 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "local", 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.960 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "selftest", 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.961 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.962 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.964 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.965 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "roption1": { 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "roption1", 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.966 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "roption2": { 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "roption2", 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "xyz", 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption1": { 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption1", 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption2": { 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption2", 2024-03-24T03:11:09.968 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption3": { 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption3", 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.969 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption4": { 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption4", 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.970 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption5": { 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption5", 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.971 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.972 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.972 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.972 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.972 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.972 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.972 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.972 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.972 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption6": { 2024-03-24T03:11:09.972 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption6", 2024-03-24T03:11:09.972 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption7": { 2024-03-24T03:11:09.973 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption7", 2024-03-24T03:11:09.974 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:09.974 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.974 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.974 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.974 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-03-24T03:11:09.974 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "42", 2024-03-24T03:11:09.974 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.974 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.974 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.974 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.974 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "testkey": { 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testkey", 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.975 INFO:teuthology.orchestra.run.smithi017.stdout: "testlkey": { 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testlkey", 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.977 INFO:teuthology.orchestra.run.smithi017.stdout: "testnewline": { 2024-03-24T03:11:09.977 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testnewline", 2024-03-24T03:11:09.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.977 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.977 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.977 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.977 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.977 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.977 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.977 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.977 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.977 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snap_schedule", 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_m_granularity": { 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_m_granularity", 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.978 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow minute scheduled snapshots", 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "dump_on_update": { 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "dump_on_update", 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.979 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "dump database to debug log on update", 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.980 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.981 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.982 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.983 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.984 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "stats", 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.985 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.986 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.987 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.988 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.989 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "status", 2024-03-24T03:11:09.990 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.991 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.992 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.993 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.994 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "telegraf", 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:09.995 INFO:teuthology.orchestra.run.smithi017.stdout: "address": { 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "address", 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:09.996 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15", 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:09.997 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.998 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:09.999 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.000 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.001 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "telemetry", 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_basic": { 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_basic", 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.002 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_crash": { 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_crash", 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.003 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_device": { 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_device", 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-24T03:11:10.004 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_ident": { 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_ident", 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_perf": { 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_perf", 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.006 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "contact": { 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "contact", 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.007 INFO:teuthology.orchestra.run.smithi017.stdout: "description": { 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "description", 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "device_url": { 2024-03-24T03:11:10.008 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_url", 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "enabled": { 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enabled", 2024-03-24T03:11:10.009 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.010 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.010 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.010 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.010 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.010 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.010 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.010 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.010 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.010 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.010 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.010 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "24", 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "8", 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.011 INFO:teuthology.orchestra.run.smithi017.stdout: "last_opt_revision": { 2024-03-24T03:11:10.012 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "last_opt_revision", 2024-03-24T03:11:10.012 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.012 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.012 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.012 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1", 2024-03-24T03:11:10.012 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.012 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.012 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.012 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.012 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.012 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.012 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "leaderboard": { 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "leaderboard", 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "leaderboard_description": { 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "leaderboard_description", 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.014 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.015 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.016 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.017 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.018 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "organization": { 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "organization", 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.019 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "proxy": { 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "proxy", 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.020 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "url": { 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "url", 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.021 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "test_orchestrator", 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "volumes", 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "max_concurrent_clones": { 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_concurrent_clones", 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "4", 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "periodic_async_work": { 2024-03-24T03:11:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "periodic_async_work", 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Periodically check for async work", 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "snapshot_clone_delay": { 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snapshot_clone_delay", 2024-03-24T03:11:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "snapshot_clone_no_wait": { 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snapshot_clone_no_wait", 2024-03-24T03:11:10.034 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix", 2024-03-24T03:11:10.035 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "discovery_interval": { 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "discovery_interval", 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "100", 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "identifier": { 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "identifier", 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.037 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.038 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.039 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.040 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.041 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.042 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_host": { 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_host", 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_port": { 2024-03-24T03:11:10.043 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_port", 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10051", 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_sender": { 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_sender", 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.044 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: "gid": 25453, 2024-03-24T03:11:10.045 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "z", 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_features": 4540701547738038271, 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "available_modules": [ 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "alerts", 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.046 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How frequently to reexamine health status", 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.047 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.048 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.049 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.050 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_destination": { 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_destination", 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.051 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Email address to send alerts to", 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_from_name": { 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_from_name", 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "Ceph", 2024-03-24T03:11:10.052 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Email From: name", 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_host": { 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_host", 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.053 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP server", 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_password": { 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_password", 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.054 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Password to authenticate with", 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_port": { 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_port", 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "465", 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP port", 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.055 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_sender": { 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_sender", 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP envelope sender", 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_ssl": { 2024-03-24T03:11:10.056 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_ssl", 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_user": { 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_user", 2024-03-24T03:11:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "User to authenticate as", 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "balancer", 2024-03-24T03:11:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "active": { 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "active", 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "automatically balance PGs across cluster", 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "begin_time": { 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "begin_time", 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0000", 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "beginning time of day to automatically balance", 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "begin_weekday": { 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "begin_weekday", 2024-03-24T03:11:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0", 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "6", 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_max_iterations": { 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_max_iterations", 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "25", 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "250", 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_metrics": { 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_metrics", 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "pgs,objects,bytes", 2024-03-24T03:11:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_step": { 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_step", 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0.5", 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0.001", 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "0.999", 2024-03-24T03:11:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "aggressiveness of optimization", 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "end_time": { 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "end_time", 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "2359", 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "ending time of day to automatically balance", 2024-03-24T03:11:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "end_weekday": { 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "end_weekday", 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0", 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "6", 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "min_score": { 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "min_score", 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": { 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mode", 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "upmap", 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-compat", 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "none", 2024-03-24T03:11:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "read", 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap", 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap-read" 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Balancer mode", 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_ids": { 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pool_ids", 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap_max_deviation": { 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "upmap_max_deviation", 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-24T03:11:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap_max_optimizations": { 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "upmap_max_optimizations", 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephadm", 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_down_multiplier": { 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_down_multiplier", 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3.0", 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_refresh_rate": { 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_refresh_rate", 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "20", 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-24T03:11:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_starting_port": { 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_starting_port", 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "4721", 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_ptrace": { 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_ptrace", 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "autotune_interval": { 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "autotune_interval", 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "autotune_memory_target_ratio": { 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "autotune_memory_target_ratio", 2024-03-24T03:11:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0.7", 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm_log_destination": { 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephadm_log_destination", 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "file", 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "file,syslog", 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "syslog" 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "cgroups_split": { 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cgroups_split", 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "config_checks_enabled": { 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_checks_enabled", 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "config_dashboard": { 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_dashboard", 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_alertmanager": { 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_alertmanager", 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_base": { 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_base", 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Container image name, without the tag", 2024-03-24T03:11:10.086 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_elasticsearch": { 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_elasticsearch", 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "elasticsearch container image", 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_grafana": { 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_grafana", 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_haproxy": { 2024-03-24T03:11:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_haproxy", 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "HAproxy container image", 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_agent": { 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_agent", 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger agent container image", 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_collector": { 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_collector", 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger collector container image", 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_query": { 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_query", 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger query container image", 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_keepalived": { 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_keepalived", 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Keepalived container image", 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_loki": { 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_loki", 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Loki container image", 2024-03-24T03:11:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_node_exporter": { 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_node_exporter", 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_nvmeof": { 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_nvmeof", 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Nvme-of container image", 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_prometheus": { 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_prometheus", 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_promtail": { 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_promtail", 2024-03-24T03:11:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Promtail container image", 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_snmp_gateway": { 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_snmp_gateway", 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SNMP Gateway container image", 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "container_init": { 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_init", 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Run podman/docker with `--init`", 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "daemon_cache_timeout": { 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "daemon_cache_timeout", 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "default_cephadm_command_timeout": { 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "default_cephadm_command_timeout", 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "900", 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "default_registry": { 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "default_registry", 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io", 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "device_cache_timeout": { 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_cache_timeout", 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1800", 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache device inventory", 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "device_enhanced_scan": { 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_enhanced_scan", 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "facts_cache_timeout": { 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "facts_cache_timeout", 2024-03-24T03:11:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache host facts data", 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "host_check_interval": { 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "host_check_interval", 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to perform a host check", 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "hw_monitoring": { 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "hw_monitoring", 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "inventory_list_all": { 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "inventory_list_all", 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "log_refresh_metadata": { 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_refresh_metadata", 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "*", 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-24T03:11:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "max_count_per_host": { 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_count_per_host", 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "max number of daemons per service per host", 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd_draining_count": { 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_osd_draining_count", 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "migration_current": { 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "migration_current", 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "internal - do not modify", 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": { 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mode", 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "root", 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm-package", 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "root" 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "mode for remote execution of cephadm", 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "oob_default_addr": { 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "oob_default_addr", 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "169.254.1.1", 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "prometheus_alerts_path": { 2024-03-24T03:11:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prometheus_alerts_path", 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_insecure": { 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_insecure", 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_password": { 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_password", 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_url": { 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_url", 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_username": { 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_username", 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-24T03:11:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "secure_monitoring_stack": { 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "secure_monitoring_stack", 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "service_discovery_port": { 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "service_discovery_port", 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8765", 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "cephadm service discovery port", 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "ssh_config_file": { 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssh_config_file", 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "use_agent": { 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "use_agent", 2024-03-24T03:11:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "use_repo_digest": { 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "use_repo_digest", 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_failed_host_check": { 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_failed_host_check", 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if the host check fails", 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_stray_daemons": { 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_stray_daemons", 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_stray_hosts": { 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_stray_hosts", 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crash", 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "retain_interval": { 2024-03-24T03:11:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "retain_interval", 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "31536000", 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_recent_interval": { 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_recent_interval", 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1209600", 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "dashboard", 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "ALERTMANAGER_API_HOST": { 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-24T03:11:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "AUDIT_API_ENABLED": { 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "AUDIT_API_ENABLED", 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "ENABLE_BROWSABLE_API": { 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-24T03:11:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_PASSWORD": { 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_URL": { 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_URL", 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_USERNAME": { 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "MULTICLUSTER_CONFIG": { 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-24T03:11:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "{}", 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "PROMETHEUS_API_HOST": { 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-24T03:11:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_ENABLED": { 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8", 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "45", 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_ACCESS_KEY": { 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-24T03:11:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-24T03:11:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_SECRET_KEY": { 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_SSL_VERIFY": { 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "UNSAFE_TLS_v1_2": { 2024-03-24T03:11:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-24T03:11:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "cross_origin_url": { 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cross_origin_url", 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "crt_file": { 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crt_file", 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "debug": { 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "debug", 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable/disable debug options", 2024-03-24T03:11:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "jwt_token_ttl": { 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "jwt_token_ttl", 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "28800", 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "key_file": { 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "key_file", 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "motd": { 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "motd", 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "The message of the day", 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "redirect_resolve_ip_addr": { 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "redirect_resolve_ip_addr", 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "::", 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8080", 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl": { 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl", 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl_server_port": { 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl_server_port", 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8443", 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_behaviour": { 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_behaviour", 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "redirect", 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "redirect" 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_error_status_code": { 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_error_status_code", 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "500", 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "400", 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "599", 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "url_prefix": { 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "url_prefix", 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "devicehealth", 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_monitoring": { 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enable_monitoring", 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "monitor device health metrics", 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "mark_out_threshold": { 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mark_out_threshold", 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "2419200", 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": { 2024-03-24T03:11:10.201 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pool_name", 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "device_health_metrics", 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "retention_period": { 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "retention_period", 2024-03-24T03:11:10.202 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15552000", 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long to retain device health metrics", 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "scrape_frequency": { 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "scrape_frequency", 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.203 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "86400", 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "self_heal": { 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "self_heal", 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.204 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.205 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to wake up and check device health", 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_threshold": { 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_threshold", 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "7257600", 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.206 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "diskprediction_local", 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.207 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.208 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.209 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.210 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "predict_interval": { 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "predict_interval", 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "86400", 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.212 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "predictor_model": { 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "predictor_model", 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "prophetstor", 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.214 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "influx", 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": false, 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "influxdb python module not found", 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: "batch_size": { 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "batch_size", 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5000", 2024-03-24T03:11:10.215 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "database": { 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "database", 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "ceph", 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.216 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": { 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "hostname", 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB server hostname", 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.217 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "30", 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "5", 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.218 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.219 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.220 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.221 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.222 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "password": { 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "password", 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.223 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "password of InfluxDB server user", 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "port": { 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "port", 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8086", 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB server port", 2024-03-24T03:11:10.224 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl": { 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl", 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "false", 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "threads": { 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "threads", 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "32", 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.226 INFO:teuthology.orchestra.run.smithi017.stdout: "username": { 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "username", 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "username of InfluxDB server user", 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "verify_ssl": { 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "verify_ssl", 2024-03-24T03:11:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "true", 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "insights", 2024-03-24T03:11:10.228 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.229 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.230 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.231 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.232 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "iostat", 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.233 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.234 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.235 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.236 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.237 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "k8sevents", 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_event_retention_days": { 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ceph_event_retention_days", 2024-03-24T03:11:10.238 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "7", 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "config_check_secs": { 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_check_secs", 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.239 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "10", 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.240 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.241 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "localpool", 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "failure_domain": { 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "failure_domain", 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "host", 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "failure domain for any created local pool", 2024-03-24T03:11:10.245 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.246 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.247 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.248 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.249 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": { 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "min_size", 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.250 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default min_size for any created local pool", 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "num_rep": { 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "num_rep", 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3", 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.251 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default replica count for any created local pool", 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": { 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pg_num", 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "128", 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.252 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default pg_num for any created local pool", 2024-03-24T03:11:10.253 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.253 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.253 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.253 INFO:teuthology.orchestra.run.smithi017.stdout: "prefix": { 2024-03-24T03:11:10.253 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prefix", 2024-03-24T03:11:10.253 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.253 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "name prefix for any created local pool", 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "subtree": { 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "subtree", 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.254 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "rack", 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mds_autoscaler", 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.255 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.256 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.258 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.259 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mirroring", 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.260 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.262 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.263 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.264 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "nfs", 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.265 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.266 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.267 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.268 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "orchestrator", 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "fail_fs": { 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "fail_fs", 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.270 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.273 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.274 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator": { 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "orchestrator", 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.275 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm", 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: "rook", 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: "test_orchestrator" 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Orchestrator backend", 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "osd_perf_query", 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.276 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.277 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.278 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.279 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.280 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "osd_support", 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.281 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.282 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.283 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.284 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.285 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pg_autoscaler", 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.286 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.288 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.289 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "threshold": { 2024-03-24T03:11:10.291 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "threshold", 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3.0", 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1.0", 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "scaling threshold", 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "progress", 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_pg_recovery_event": { 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_pg_recovery_event", 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.293 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "enabled": { 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enabled", 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.295 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.296 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "max_completed_events": { 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_completed_events", 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "50", 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "number of past completed events to remember", 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long the module is going to sleep", 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prometheus", 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "cache": { 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cache", 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "exclude_perf_counters": { 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "exclude_perf_counters", 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_stats_pools": { 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_stats_pools", 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "300", 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "scrape_interval": { 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "scrape_interval", 2024-03-24T03:11:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15.0", 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "::", 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "9283", 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "stale_cache_strategy": { 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "stale_cache_strategy", 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "log", 2024-03-24T03:11:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_behaviour": { 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_behaviour", 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "default", 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "default", 2024-03-24T03:11:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "error" 2024-03-24T03:11:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_error_status_code": { 2024-03-24T03:11:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_error_status_code", 2024-03-24T03:11:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "500", 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "400", 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "599", 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_support", 2024-03-24T03:11:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "max_concurrent_snap_create": { 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_concurrent_snap_create", 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.321 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.321 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.321 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.321 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.321 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.321 INFO:teuthology.orchestra.run.smithi017.stdout: "mirror_snapshot_schedule": { 2024-03-24T03:11:10.321 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mirror_snapshot_schedule", 2024-03-24T03:11:10.321 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.322 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.322 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.322 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.322 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.322 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.322 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.322 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.322 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.323 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.323 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.323 INFO:teuthology.orchestra.run.smithi017.stdout: "trash_purge_schedule": { 2024-03-24T03:11:10.323 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "trash_purge_schedule", 2024-03-24T03:11:10.323 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.323 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.323 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.323 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.324 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.324 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.324 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.324 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.324 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.324 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.324 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.324 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.324 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.325 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "restful", 2024-03-24T03:11:10.325 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.325 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.325 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.325 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_auth": { 2024-03-24T03:11:10.325 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enable_auth", 2024-03-24T03:11:10.325 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "key_file": { 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "key_file", 2024-03-24T03:11:10.326 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.327 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.329 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.330 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.331 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.332 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.333 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rgw", 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.334 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.335 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.336 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.337 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rook", 2024-03-24T03:11:10.338 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.339 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.340 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.341 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.342 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "storage_class": { 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "storage_class", 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.343 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "local", 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "selftest", 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.344 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.345 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.346 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.347 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.348 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "roption1": { 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "roption1", 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "roption2": { 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "roption2", 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "xyz", 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.350 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption1": { 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption1", 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.351 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption2": { 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption2", 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption3": { 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption3", 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.352 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption4": { 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption4", 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.353 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption5": { 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption5", 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.354 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption6": { 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption6", 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.355 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption7": { 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption7", 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "42", 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.356 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "testkey": { 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testkey", 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.357 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "testlkey": { 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testlkey", 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.358 INFO:teuthology.orchestra.run.smithi017.stdout: "testnewline": { 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testnewline", 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snap_schedule", 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_m_granularity": { 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_m_granularity", 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow minute scheduled snapshots", 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.360 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "dump_on_update": { 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "dump_on_update", 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "dump database to debug log on update", 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.361 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.362 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.363 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.364 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.365 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "stats", 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.366 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.367 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.368 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.369 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.370 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "status", 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.371 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.372 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.373 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "telegraf", 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "address": { 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "address", 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.376 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15", 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.377 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.378 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.379 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.380 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.381 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "telemetry", 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.382 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_basic": { 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_basic", 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_crash": { 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_crash", 2024-03-24T03:11:10.383 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_device": { 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_device", 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.384 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_ident": { 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_ident", 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.385 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_perf": { 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_perf", 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.386 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "contact": { 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "contact", 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.387 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "description": { 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "description", 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.388 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "device_url": { 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_url", 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.389 INFO:teuthology.orchestra.run.smithi017.stdout: "enabled": { 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enabled", 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:10.390 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "24", 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "8", 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "last_opt_revision": { 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "last_opt_revision", 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.391 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1", 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "leaderboard": { 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "leaderboard", 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.392 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "leaderboard_description": { 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "leaderboard_description", 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.393 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.394 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.395 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.396 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.397 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "organization": { 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "organization", 2024-03-24T03:11:10.398 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "proxy": { 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "proxy", 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.399 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "url": { 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "url", 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-24T03:11:10.400 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "test_orchestrator", 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.401 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.402 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.403 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.404 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.405 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "volumes", 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.406 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.407 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.410 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "max_concurrent_clones": { 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_concurrent_clones", 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "4", 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-24T03:11:10.411 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "periodic_async_work": { 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "periodic_async_work", 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Periodically check for async work", 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.412 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "snapshot_clone_delay": { 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snapshot_clone_delay", 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "snapshot_clone_no_wait": { 2024-03-24T03:11:10.413 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snapshot_clone_no_wait", 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix", 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "discovery_interval": { 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "discovery_interval", 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "100", 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.415 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "identifier": { 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "identifier", 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.416 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.417 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.418 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.419 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.420 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.421 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_host": { 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_host", 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.422 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_port": { 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_port", 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10051", 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_sender": { 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_sender", 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.424 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: "modules": [ 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm", 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: "dashboard", 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: "iostat", 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: "nfs", 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: "restful", 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_schedule" 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: "available_modules": [ 2024-03-24T03:11:10.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "alerts", 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How frequently to reexamine health status", 2024-03-24T03:11:10.426 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.427 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.428 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.429 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.430 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_destination": { 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_destination", 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.431 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Email address to send alerts to", 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_from_name": { 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_from_name", 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "Ceph", 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.432 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Email From: name", 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_host": { 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_host", 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP server", 2024-03-24T03:11:10.433 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_password": { 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_password", 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Password to authenticate with", 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.434 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_port": { 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_port", 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "465", 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP port", 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.435 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_sender": { 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_sender", 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP envelope sender", 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_ssl": { 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_ssl", 2024-03-24T03:11:10.436 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_user": { 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_user", 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.437 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "User to authenticate as", 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "balancer", 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: "active": { 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "active", 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "automatically balance PGs across cluster", 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "begin_time": { 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "begin_time", 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0000", 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "beginning time of day to automatically balance", 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "begin_weekday": { 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "begin_weekday", 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:10.440 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0", 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "6", 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_max_iterations": { 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_max_iterations", 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "25", 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-03-24T03:11:10.441 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "250", 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_metrics": { 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_metrics", 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "pgs,objects,bytes", 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.442 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_step": { 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_step", 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0.5", 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0.001", 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "0.999", 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "aggressiveness of optimization", 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-24T03:11:10.443 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "end_time": { 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "end_time", 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "2359", 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "ending time of day to automatically balance", 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.444 INFO:teuthology.orchestra.run.smithi017.stdout: "end_weekday": { 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "end_weekday", 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0", 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "6", 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.445 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.446 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.447 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.448 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.449 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "min_score": { 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "min_score", 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": { 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mode", 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "upmap", 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-compat", 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "none", 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "read", 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap", 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap-read" 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.451 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Balancer mode", 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_ids": { 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pool_ids", 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.452 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.453 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap_max_deviation": { 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "upmap_max_deviation", 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap_max_optimizations": { 2024-03-24T03:11:10.454 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "upmap_max_optimizations", 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephadm", 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_down_multiplier": { 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_down_multiplier", 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3.0", 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.456 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_refresh_rate": { 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_refresh_rate", 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "20", 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_starting_port": { 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_starting_port", 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "4721", 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_ptrace": { 2024-03-24T03:11:10.458 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_ptrace", 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "autotune_interval": { 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "autotune_interval", 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.459 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "autotune_memory_target_ratio": { 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "autotune_memory_target_ratio", 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0.7", 2024-03-24T03:11:10.460 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm_log_destination": { 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephadm_log_destination", 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.461 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "file", 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "file,syslog", 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "syslog" 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "cgroups_split": { 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cgroups_split", 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.462 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "config_checks_enabled": { 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_checks_enabled", 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.463 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "config_dashboard": { 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_dashboard", 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-24T03:11:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_alertmanager": { 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_alertmanager", 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_base": { 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_base", 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Container image name, without the tag", 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_elasticsearch": { 2024-03-24T03:11:10.466 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_elasticsearch", 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "elasticsearch container image", 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_grafana": { 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_grafana", 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.467 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_haproxy": { 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_haproxy", 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.468 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "HAproxy container image", 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_agent": { 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_agent", 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.469 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger agent container image", 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_collector": { 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_collector", 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.470 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger collector container image", 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_query": { 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_query", 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger query container image", 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.471 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_keepalived": { 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_keepalived", 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Keepalived container image", 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.472 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_loki": { 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_loki", 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Loki container image", 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_node_exporter": { 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_node_exporter", 2024-03-24T03:11:10.473 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_nvmeof": { 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_nvmeof", 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.474 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Nvme-of container image", 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_prometheus": { 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_prometheus", 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-24T03:11:10.475 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_promtail": { 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_promtail", 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.476 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Promtail container image", 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_snmp_gateway": { 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_snmp_gateway", 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SNMP Gateway container image", 2024-03-24T03:11:10.477 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "container_init": { 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_init", 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Run podman/docker with `--init`", 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.478 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "daemon_cache_timeout": { 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "daemon_cache_timeout", 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.479 INFO:teuthology.orchestra.run.smithi017.stdout: "default_cephadm_command_timeout": { 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "default_cephadm_command_timeout", 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "900", 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "default_registry": { 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "default_registry", 2024-03-24T03:11:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io", 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "device_cache_timeout": { 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_cache_timeout", 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.481 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1800", 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache device inventory", 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "device_enhanced_scan": { 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_enhanced_scan", 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.482 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "facts_cache_timeout": { 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "facts_cache_timeout", 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.483 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache host facts data", 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "host_check_interval": { 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "host_check_interval", 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to perform a host check", 2024-03-24T03:11:10.484 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "hw_monitoring": { 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "hw_monitoring", 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.485 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "inventory_list_all": { 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "inventory_list_all", 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.486 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.487 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "log_refresh_metadata": { 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_refresh_metadata", 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.488 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.489 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.490 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.491 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.491 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.491 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.491 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.491 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.491 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.491 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.491 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.491 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.492 INFO:teuthology.orchestra.run.smithi017.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-24T03:11:10.493 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "*", 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "max_count_per_host": { 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_count_per_host", 2024-03-24T03:11:10.494 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "max number of daemons per service per host", 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd_draining_count": { 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_osd_draining_count", 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.495 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "migration_current": { 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "migration_current", 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.496 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "internal - do not modify", 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": { 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mode", 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "root", 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm-package", 2024-03-24T03:11:10.497 INFO:teuthology.orchestra.run.smithi017.stdout: "root" 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "mode for remote execution of cephadm", 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "oob_default_addr": { 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "oob_default_addr", 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "169.254.1.1", 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.498 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "prometheus_alerts_path": { 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prometheus_alerts_path", 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.499 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_insecure": { 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_insecure", 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_password": { 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_password", 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_url": { 2024-03-24T03:11:10.501 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_url", 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_username": { 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_username", 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.502 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "secure_monitoring_stack": { 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "secure_monitoring_stack", 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.503 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "service_discovery_port": { 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "service_discovery_port", 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8765", 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.504 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "cephadm service discovery port", 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "ssh_config_file": { 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssh_config_file", 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-24T03:11:10.505 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "use_agent": { 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "use_agent", 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.506 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "use_repo_digest": { 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "use_repo_digest", 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.507 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_failed_host_check": { 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_failed_host_check", 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if the host check fails", 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_stray_daemons": { 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_stray_daemons", 2024-03-24T03:11:10.508 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_stray_hosts": { 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_stray_hosts", 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.509 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crash", 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.511 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.512 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.513 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.514 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "retain_interval": { 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "retain_interval", 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "31536000", 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.515 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_recent_interval": { 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_recent_interval", 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1209600", 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.516 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "dashboard", 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.517 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "ALERTMANAGER_API_HOST": { 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.518 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.519 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "AUDIT_API_ENABLED": { 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "AUDIT_API_ENABLED", 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.520 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "ENABLE_BROWSABLE_API": { 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.522 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-24T03:11:10.523 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-24T03:11:10.524 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-24T03:11:10.525 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-24T03:11:10.526 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-24T03:11:10.527 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.528 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.529 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_PASSWORD": { 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.530 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-24T03:11:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_URL": { 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_URL", 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.532 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_USERNAME": { 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.533 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.534 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.535 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.536 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.537 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "MULTICLUSTER_CONFIG": { 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "{}", 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.538 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "PROMETHEUS_API_HOST": { 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.539 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.540 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-24T03:11:10.544 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-24T03:11:10.545 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-24T03:11:10.546 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_ENABLED": { 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-24T03:11:10.547 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.548 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.549 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8", 2024-03-24T03:11:10.550 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "45", 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.551 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_ACCESS_KEY": { 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.552 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.553 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_SECRET_KEY": { 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.554 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_SSL_VERIFY": { 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.555 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "UNSAFE_TLS_v1_2": { 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.556 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-24T03:11:10.557 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-24T03:11:10.558 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "cross_origin_url": { 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cross_origin_url", 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.559 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "crt_file": { 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crt_file", 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.560 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "debug": { 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "debug", 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.561 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable/disable debug options", 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "jwt_token_ttl": { 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "jwt_token_ttl", 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "28800", 2024-03-24T03:11:10.562 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "key_file": { 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "key_file", 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.563 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.564 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.565 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.566 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.567 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "motd": { 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "motd", 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.568 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "The message of the day", 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "redirect_resolve_ip_addr": { 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "redirect_resolve_ip_addr", 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.569 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.570 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "::", 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8080", 2024-03-24T03:11:10.571 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl": { 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl", 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.572 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl_server_port": { 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl_server_port", 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8443", 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.573 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_behaviour": { 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_behaviour", 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "redirect", 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "redirect" 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.574 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_error_status_code": { 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_error_status_code", 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "500", 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "400", 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "599", 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.575 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "url_prefix": { 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "url_prefix", 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.576 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "devicehealth", 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_monitoring": { 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enable_monitoring", 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "monitor device health metrics", 2024-03-24T03:11:10.577 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.578 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.579 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.580 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.581 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "mark_out_threshold": { 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mark_out_threshold", 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "2419200", 2024-03-24T03:11:10.582 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": { 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pool_name", 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "device_health_metrics", 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.583 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "retention_period": { 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "retention_period", 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15552000", 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long to retain device health metrics", 2024-03-24T03:11:10.584 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "scrape_frequency": { 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "scrape_frequency", 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "86400", 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "self_heal": { 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "self_heal", 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:10.586 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to wake up and check device health", 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_threshold": { 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_threshold", 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.587 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "7257600", 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "diskprediction_local", 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.588 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.589 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.590 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.591 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.592 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "predict_interval": { 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "predict_interval", 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "86400", 2024-03-24T03:11:10.593 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: "predictor_model": { 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "predictor_model", 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.594 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "prophetstor", 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.595 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "influx", 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": false, 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "influxdb python module not found", 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: "batch_size": { 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "batch_size", 2024-03-24T03:11:10.596 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5000", 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "database": { 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "database", 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.597 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "ceph", 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": { 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "hostname", 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.598 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB server hostname", 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "30", 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "5", 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.599 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.600 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.601 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.602 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.603 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "password": { 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "password", 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.604 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "password of InfluxDB server user", 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "port": { 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "port", 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8086", 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB server port", 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl": { 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl", 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "false", 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-24T03:11:10.606 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "threads": { 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "threads", 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "32", 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "username": { 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "username", 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "username of InfluxDB server user", 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "verify_ssl": { 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "verify_ssl", 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "true", 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.609 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "insights", 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.610 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.611 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.612 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.613 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.614 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "iostat", 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.615 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.616 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.617 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.618 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.619 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "k8sevents", 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_event_retention_days": { 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ceph_event_retention_days", 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "7", 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.620 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "config_check_secs": { 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_check_secs", 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "10", 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-24T03:11:10.621 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.622 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "localpool", 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "failure_domain": { 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "failure_domain", 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "host", 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "failure domain for any created local pool", 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": { 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "min_size", 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default min_size for any created local pool", 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "num_rep": { 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "num_rep", 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3", 2024-03-24T03:11:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default replica count for any created local pool", 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": { 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pg_num", 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "128", 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default pg_num for any created local pool", 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "prefix": { 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prefix", 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "name prefix for any created local pool", 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "subtree": { 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "subtree", 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "rack", 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mds_autoscaler", 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mirroring", 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "nfs", 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "orchestrator", 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "fail_fs": { 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "fail_fs", 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator": { 2024-03-24T03:11:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "orchestrator", 2024-03-24T03:11:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm", 2024-03-24T03:11:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "rook", 2024-03-24T03:11:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "test_orchestrator" 2024-03-24T03:11:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Orchestrator backend", 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "osd_perf_query", 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "osd_support", 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pg_autoscaler", 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.808 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:10.809 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "threshold": { 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "threshold", 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.810 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3.0", 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1.0", 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "scaling threshold", 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.811 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "progress", 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_pg_recovery_event": { 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_pg_recovery_event", 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.812 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: "enabled": { 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enabled", 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.813 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.814 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.816 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.817 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.818 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "max_completed_events": { 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_completed_events", 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "50", 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "number of past completed events to remember", 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.819 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long the module is going to sleep", 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.820 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prometheus", 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "cache": { 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cache", 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.821 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: "exclude_perf_counters": { 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "exclude_perf_counters", 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.822 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.823 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.825 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.826 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.827 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_stats_pools": { 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_stats_pools", 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.828 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "300", 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.829 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: "scrape_interval": { 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "scrape_interval", 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15.0", 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.830 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "::", 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-24T03:11:10.831 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "9283", 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.832 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "stale_cache_strategy": { 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "stale_cache_strategy", 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "log", 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_behaviour": { 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_behaviour", 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "default", 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: "default", 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: "error" 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.834 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_error_status_code": { 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_error_status_code", 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "500", 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "400", 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "599", 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.835 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_support", 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.836 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "max_concurrent_snap_create": { 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_concurrent_snap_create", 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "mirror_snapshot_schedule": { 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mirror_snapshot_schedule", 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "trash_purge_schedule": { 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "trash_purge_schedule", 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "restful", 2024-03-24T03:11:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_auth": { 2024-03-24T03:11:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enable_auth", 2024-03-24T03:11:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "key_file": { 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "key_file", 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-03-24T03:11:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.852 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rgw", 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.853 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.854 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.855 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.856 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.857 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.858 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rook", 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.859 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.860 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.861 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.862 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.862 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.862 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.862 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.862 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.862 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.862 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.862 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.862 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.862 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.862 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.862 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.863 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "storage_class": { 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "storage_class", 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "local", 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-24T03:11:10.864 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "selftest", 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.865 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.866 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.867 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.868 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.869 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "roption1": { 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "roption1", 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.870 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "roption2": { 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "roption2", 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "xyz", 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.871 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption1": { 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption1", 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.872 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption2": { 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption2", 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.873 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption3": { 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption3", 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.874 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption4": { 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption4", 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption5": { 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption5", 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption6": { 2024-03-24T03:11:10.876 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption6", 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption7": { 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption7", 2024-03-24T03:11:10.877 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "42", 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: "testkey": { 2024-03-24T03:11:10.878 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testkey", 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "testlkey": { 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testlkey", 2024-03-24T03:11:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "testnewline": { 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testnewline", 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.880 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snap_schedule", 2024-03-24T03:11:10.881 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_m_granularity": { 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_m_granularity", 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow minute scheduled snapshots", 2024-03-24T03:11:10.882 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "dump_on_update": { 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "dump_on_update", 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "dump database to debug log on update", 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.883 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.884 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.885 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.886 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "stats", 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.889 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.890 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.893 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "status", 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.894 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.895 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.896 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.897 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.898 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "telegraf", 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: "address": { 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "address", 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-24T03:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15", 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.900 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.901 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.902 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.903 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.904 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.905 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "telemetry", 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_basic": { 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_basic", 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.906 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_crash": { 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_crash", 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.907 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_device": { 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_device", 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.908 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_ident": { 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_ident", 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.909 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_perf": { 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_perf", 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.910 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "contact": { 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "contact", 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.911 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "description": { 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "description", 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.912 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "device_url": { 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_url", 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.913 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "enabled": { 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enabled", 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.914 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "24", 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "8", 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.915 INFO:teuthology.orchestra.run.smithi017.stdout: "last_opt_revision": { 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "last_opt_revision", 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1", 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "leaderboard": { 2024-03-24T03:11:10.916 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "leaderboard", 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "leaderboard_description": { 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "leaderboard_description", 2024-03-24T03:11:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.918 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.920 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.921 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.922 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "organization": { 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "organization", 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.923 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "proxy": { 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "proxy", 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.924 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "url": { 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "url", 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-24T03:11:10.925 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "test_orchestrator", 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.926 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.927 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.928 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.929 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.930 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "volumes", 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.931 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.932 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.933 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.934 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.935 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "max_concurrent_clones": { 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_concurrent_clones", 2024-03-24T03:11:10.936 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "4", 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "periodic_async_work": { 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "periodic_async_work", 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.937 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Periodically check for async work", 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "snapshot_clone_delay": { 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snapshot_clone_delay", 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.938 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "snapshot_clone_no_wait": { 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snapshot_clone_no_wait", 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-03-24T03:11:10.939 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix", 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: "discovery_interval": { 2024-03-24T03:11:10.940 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "discovery_interval", 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "100", 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "identifier": { 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "identifier", 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.941 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-03-24T03:11:10.942 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.944 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-03-24T03:11:10.945 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.946 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.947 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_host": { 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_host", 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_port": { 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_port", 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10051", 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_sender": { 2024-03-24T03:11:10.949 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_sender", 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.950 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "services": { 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "dashboard": "https://172.21.15.17:8443/" 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "always_on_modules": { 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus": [ 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "balancer", 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "crash", 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "devicehealth", 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator", 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscaler", 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "progress", 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_support", 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "status", 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "telemetry", 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: "volumes" 2024-03-24T03:11:10.951 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific": [ 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "balancer", 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "crash", 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "devicehealth", 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator", 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscaler", 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "progress", 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_support", 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "status", 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "telemetry", 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "volumes" 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy": [ 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "balancer", 2024-03-24T03:11:10.952 INFO:teuthology.orchestra.run.smithi017.stdout: "crash", 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "devicehealth", 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator", 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscaler", 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "progress", 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_support", 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "status", 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "telemetry", 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "volumes" 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "reef": [ 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "balancer", 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "crash", 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "devicehealth", 2024-03-24T03:11:10.953 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator", 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscaler", 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "progress", 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_support", 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "status", 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "telemetry", 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "volumes" 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "squid": [ 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "balancer", 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "crash", 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "devicehealth", 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator", 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscaler", 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "progress", 2024-03-24T03:11:10.954 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_support", 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: "status", 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: "telemetry", 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: "volumes" 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: "last_failure_osd_epoch": 94, 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: "active_clients": [ 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "devicehealth", 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:0", 2024-03-24T03:11:10.955 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2829664829 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "libcephsqlite", 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:0", 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3598474291 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_support", 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-03-24T03:11:10.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:0", 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2741402102 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snap_schedule", 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:0", 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4080998589 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-24T03:11:10.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snap_schedule", 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:0", 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4267732811 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "volumes", 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-03-24T03:11:10.958 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:0", 2024-03-24T03:11:10.959 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 96580409 2024-03-24T03:11:10.959 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.959 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-24T03:11:10.959 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:10.959 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-24T03:11:10.959 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:11:10.960 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:10 smithi019 ceph-mon[35944]: pgmap v313: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:11:10.960 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:10 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3177474508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:11:10.960 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:10 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/796736014' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-24T03:11:10.960 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:10 smithi188 ceph-mon[36206]: pgmap v313: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:11:10.960 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:10 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3177474508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:11:10.961 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:10 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/796736014' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-24T03:11:10.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.644+0000 7f6c7ca61640 1 -- 172.21.15.17:0/796736014 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f6c640a29f0 msgr2=0x7f6c640a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:10.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.644+0000 7f6c7ca61640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f6c640a29f0 0x7f6c640a4eb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6c6802efd0 tx=0x7f6c6802cf90 comp rx=0 tx=0).stop 2024-03-24T03:11:10.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.644+0000 7f6c7ca61640 1 -- 172.21.15.17:0/796736014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7810bca0 msgr2=0x7f6c7819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:10.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.644+0000 7f6c7ca61640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7810bca0 0x7f6c7819c7f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000eb40 tx=0x7f6c7000c2d0 comp rx=0 tx=0).stop 2024-03-24T03:11:10.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.644+0000 7f6c7ca61640 1 -- 172.21.15.17:0/796736014 shutdown_connections 2024-03-24T03:11:10.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.644+0000 7f6c7ca61640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f6c640a29f0 0x7f6c640a4eb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:10.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.644+0000 7f6c7ca61640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7810bca0 0x7f6c7819c7f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:10.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.644+0000 7f6c7ca61640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c781031f0 0x7f6c78198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:10.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.644+0000 7f6c7ca61640 1 --2- 172.21.15.17:0/796736014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c781028d0 0x7f6c78197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:10.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.644+0000 7f6c7ca61640 1 -- 172.21.15.17:0/796736014 >> 172.21.15.17:0/796736014 conn(0x7f6c78076920 msgr2=0x7f6c780fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:10.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.644+0000 7f6c7ca61640 1 -- 172.21.15.17:0/796736014 shutdown_connections 2024-03-24T03:11:10.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:09.644+0000 7f6c7ca61640 1 -- 172.21.15.17:0/796736014 wait complete. 2024-03-24T03:11:10.967 DEBUG:tasks.check_counter:Getting stats from a 2024-03-24T03:11:10.968 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-24T03:11:10.968 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:10 smithi017 ceph-mon[31926]: pgmap v313: 129 pgs: 129 active+clean; 12 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-24T03:11:10.968 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:10 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3177474508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:11:10.968 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:10 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/796736014' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-24T03:11:11.225 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:11:11.347 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:11 smithi017 ceph-mon[31926]: pgmap v314: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-03-24T03:11:11.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:11 smithi188 ceph-mon[36206]: pgmap v314: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-03-24T03:11:11.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:11 smithi019 ceph-mon[35944]: pgmap v314: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-03-24T03:11:13.093 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:11:13.093 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-0": { 2024-03-24T03:11:13.093 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 969, 2024-03-24T03:11:13.093 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 958, 2024-03-24T03:11:13.093 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 1715136, 2024-03-24T03:11:13.093 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 359910, 2024-03-24T03:11:13.093 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 5, 2024-03-24T03:11:13.093 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 1, 2024-03-24T03:11:13.093 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.131014958, 2024-03-24T03:11:13.093 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.044130893, 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.055350614, 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.030782799, 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 957, 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.052992354, 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000055373 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 1468848, 2024-03-24T03:11:13.094 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 126816 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-1": { 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 681, 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 834, 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 170094, 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 648318, 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 7, 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 5, 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.077793949, 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.026804274, 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.017588783, 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.026050781, 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:13.095 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 832, 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.038754728, 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000046580 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 433600 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-2": { 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 740, 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 742, 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 195785, 2024-03-24T03:11:13.096 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 235321, 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 4, 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 3, 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.064167247, 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.022510790, 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.016417667, 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.019164541, 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 740, 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.025827996, 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000034902 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 0 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: "cct": { 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: "total_workers": 1, 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: "unhealthy_workers": 0 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: "finisher-MDSRank": { 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: "queue_len": 0, 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: "complete_latency": { 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 3482, 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.108912688, 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000031278 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:13.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "finisher-PurgeQueue": { 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "queue_len": 0, 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "complete_latency": { 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": { 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "request": 0, 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "reply": 0, 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "reply_latency": { 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "slow_reply": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "forward": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_fetch_complete": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_fetch_keys": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_commit": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_split": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_merge": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_top": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_bottom": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_pin_tail": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_pinned": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_expired": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_with_caps": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": 0, 2024-03-24T03:11:13.100 INFO:teuthology.orchestra.run.smithi017.stdout: "subtrees": 1, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_hit": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_forward": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_discover": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_dir_fetch": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_remote_ino": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_lock": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "load_cent": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "q": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "exported": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "exported_inodes": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "imported": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "imported_inodes": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_dir_fetch": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_backtrace_fetch": 0, 2024-03-24T03:11:13.101 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_peer_discover": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rfiles": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rbytes": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rsnaps": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_backtrace_fetch": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_set_tag": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_backtrace_repaired": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_inotable_repaired": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dir_inodes": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dir_base_inodes": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dirfrag_rstats": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_file_inodes": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_inode_file_caps": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_revoke": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_grant": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_trunc": 0, 2024-03-24T03:11:13.102 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_caps": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_caps_dirty": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_cap_release": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "process_request_cap_release": 0 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache": { 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_update": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_update_receipt": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_try_discover": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_send_discover": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_handle_discover": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays_delayed": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays_enqueuing": 0, 2024-03-24T03:11:13.103 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_created": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_enqueued": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_reintegrated": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_migrated": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_processing": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_enqueued": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_prioritized": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_started": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_completed": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_enqueue_scrub": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_exportdir": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_flush": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_fragmentdir": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_fragstats": 0, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_inodestats": 0 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log": { 2024-03-24T03:11:13.104 INFO:teuthology.orchestra.run.smithi017.stdout: "evlrg": 0, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "evadd": 0, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "evex": 0, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "evtrm": 0, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "ev": 7, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "evexg": 0, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "evexd": 0, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "segadd": 0, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "segex": 0, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "segtrm": 0, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "seg": 2, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "segmjr": 2, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "segexg": 0, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "segexd": 0, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "expos": 4194304, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "wrpos": 4196264, 2024-03-24T03:11:13.105 INFO:teuthology.orchestra.run.smithi017.stdout: "rdpos": 4196123, 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "jlat": { 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "replayed": 7 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_mem": { 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 3, 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "ino+": 3, 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "ino-": 0, 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "dir": 1, 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "dir+": 1, 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "dir-": 0, 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "dn": 0, 2024-03-24T03:11:13.106 INFO:teuthology.orchestra.run.smithi017.stdout: "dn+": 0, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "dn-": 0, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "cap": 0, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "cap+": 0, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "cap-": 0, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "rss": 50524, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "heap": 215420 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_server": { 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "dispatch_client_request": 0, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "dispatch_server_request": 0, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_request": 0, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_session": 0, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_peer_request": 0, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "req_create_latency": { 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.107 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getattr_latency": { 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getfilelock_latency": { 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: "req_link_latency": { 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookup_latency": { 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookuphash_latency": { 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupino_latency": { 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupname_latency": { 2024-03-24T03:11:13.109 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupparent_latency": { 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupsnap_latency": { 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lssnap_latency": { 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.110 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.111 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.111 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mkdir_latency": { 2024-03-24T03:11:13.111 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.111 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.111 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.111 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mknod_latency": { 2024-03-24T03:11:13.111 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mksnap_latency": { 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "req_open_latency": { 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "req_readdir_latency": { 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rename_latency": { 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: "req_renamesnap_latency": { 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: "req_snapdiff_latency": { 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmdir_latency": { 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmsnap_latency": { 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmxattr_latency": { 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setattr_latency": { 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setdirlayout_latency": { 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setfilelock_latency": { 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.115 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setlayout_latency": { 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setxattr_latency": { 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "req_symlink_latency": { 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "req_unlink_latency": { 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.116 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "cap_revoke_eviction": 0, 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "cap_acquisition_throttle": 0, 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getvxattr_latency": { 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_sessions": { 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "session_count": 1, 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "session_add": 3, 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "session_remove": 2, 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "sessions_open": 2, 2024-03-24T03:11:13.117 INFO:teuthology.orchestra.run.smithi017.stdout: "sessions_stale": 0, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "total_load": 0, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "average_load": 0, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "avg_session_uptime": 2942, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "mdthresh_evicted": 0 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool": { 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_bytes": 0, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_items": 0, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_bytes": 0, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_items": 0, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_bytes": 0, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_items": 0, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_items": 0, 2024-03-24T03:11:13.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_items": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_bytes": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_items": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_items": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_bytes": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_items": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_bytes": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_items": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_items": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_items": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_bytes": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_items": 0, 2024-03-24T03:11:13.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_bytes": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_items": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_items": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_bytes": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_items": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_bytes": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_items": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_bytes": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_items": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_bytes": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_items": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_bytes": 189433, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_items": 52, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_bytes": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_items": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bytes": 0, 2024-03-24T03:11:13.120 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_items": 0, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_bytes": 0, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_items": 0, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_bytes": 0, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_items": 0, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_bytes": 10480, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_items": 222, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_bytes": 0, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_items": 0, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_bytes": 0, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_items": 0, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_bytes": 8320, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_items": 13, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_bytes": 0, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_items": 0, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_bytes": 0, 2024-03-24T03:11:13.121 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_items": 0 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter": { 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "op_active": 0, 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "op_laggy": 0, 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "op_send": 2148, 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "op_send_bytes": 21, 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "op_resend": 3, 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "op_reply": 2148, 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "op_latency": { 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 2148, 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 1.101987240, 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000513029 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "op_inflight": 0, 2024-03-24T03:11:13.122 INFO:teuthology.orchestra.run.smithi017.stdout: "oplen_avg": { 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 2148, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 2149 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "op": 2148, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "op_r": 2148, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "op_w": 0, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "op_rmw": 0, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "op_pg": 0, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_stat": 1405, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_create": 0, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_read": 741, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_write": 0, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_writefull": 0, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_writesame": 0, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_append": 0, 2024-03-24T03:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_zero": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_truncate": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_delete": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_mapext": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_sparse_read": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_clonerange": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_getxattr": 1, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_setxattr": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_cmpxattr": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_rmxattr": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_resetxattrs": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_call": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_watch": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_notify": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_src_cmpxattr": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_pgls": 0, 2024-03-24T03:11:13.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_pgls_filter": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_other": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_active": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_send": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_resend": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_ping": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_active": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_send": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_resend": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_active": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_send": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_resend": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_active": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_send": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_resend": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "command_active": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "command_send": 0, 2024-03-24T03:11:13.125 INFO:teuthology.orchestra.run.smithi017.stdout: "command_resend": 0, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "map_epoch": 102, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "map_full": 0, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "map_inc": 13, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_sessions": 5, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_session_open": 5, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_session_close": 0, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_laggy": 0, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_wr": 0, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_rd": 2, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_del": 0 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "oft": { 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_objs": 0, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_kv_pairs": 0, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_updates": 0, 2024-03-24T03:11:13.126 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_removes": 0 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "osd-slow-ops": { 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "slow_ops_count": 0 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "purge_queue": { 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_ops": 0, 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_ops_high_water": 0, 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing": 0, 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_high_water": 0, 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executed_ops": 0, 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executed": 0, 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_item_in_journal": 0 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-03-24T03:11:13.127 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 2386, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 1851117, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 2386, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 2386, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 1851117, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-objecter_bytes": { 2024-03-24T03:11:13.128 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 2148, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 1965, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 2148, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 1965, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:13.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-objecter_ops": { 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 1024, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 2148, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 2148, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 2148, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 2148, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.130 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-write_buf_throttle": { 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 3758096384, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 0, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 0, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.131 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-write_buf_throttle-0x55a0ab6a65a0": { 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 3758096384, 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 0, 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 0, 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-03-24T03:11:13.132 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:13.133 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:13.133 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:13.133 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:13.133 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:13.133 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:11:13.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:13 smithi017 ceph-mon[31926]: pgmap v315: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 40 KiB/s wr, 32 op/s 2024-03-24T03:11:13.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:13 smithi188 ceph-mon[36206]: pgmap v315: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 40 KiB/s wr, 32 op/s 2024-03-24T03:11:13.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:13 smithi019 ceph-mon[35944]: pgmap v315: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 40 KiB/s wr, 32 op/s 2024-03-24T03:11:14.223 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-24T03:11:14.223 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-24T03:11:14.223 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-03-24T03:11:14.223 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-03-24T03:11:14.223 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-03-24T03:11:14.223 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-24T03:11:14.223 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-24T03:11:14.224 DEBUG:tasks.check_counter:Getting stats from d 2024-03-24T03:11:14.224 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-24T03:11:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:11:15.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:15 smithi017 ceph-mon[31926]: pgmap v316: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 40 KiB/s wr, 32 op/s 2024-03-24T03:11:15.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:15 smithi188 ceph-mon[36206]: pgmap v316: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 40 KiB/s wr, 32 op/s 2024-03-24T03:11:15.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:15 smithi019 ceph-mon[35944]: pgmap v316: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 40 KiB/s wr, 32 op/s 2024-03-24T03:11:16.291 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:11:16.291 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-0": { 2024-03-24T03:11:16.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 4, 2024-03-24T03:11:16.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 4, 2024-03-24T03:11:16.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 3072, 2024-03-24T03:11:16.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 448, 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 4, 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 0, 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.001609087, 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.000042619, 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.000758505, 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 3, 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000816241, 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000272080 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:16.292 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 448 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-1": { 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 2, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 162, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 320, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 183824, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 5, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 3, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.013907388, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.009469692, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.001689692, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-24T03:11:16.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 159, 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.009550478, 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000060065 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 183824 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-2": { 2024-03-24T03:11:16.294 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 223, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 213, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 1437984, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 127440, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 2, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 1, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.042375974, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.014242564, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.031343631, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.004621941, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 212, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.014071183, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000066373 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:16.295 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 1437984, 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 127440 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "cct": { 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "total_workers": 0, 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "unhealthy_workers": 0 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool": { 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_bytes": 0, 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_items": 0, 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_bytes": 0, 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_items": 0, 2024-03-24T03:11:16.296 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_bytes": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_items": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_items": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_items": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_bytes": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_items": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_items": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_bytes": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_items": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_bytes": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_items": 0, 2024-03-24T03:11:16.297 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_items": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_items": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_bytes": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_items": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_bytes": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_items": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_items": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_bytes": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_items": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_bytes": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_items": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_bytes": 0, 2024-03-24T03:11:16.298 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_items": 0, 2024-03-24T03:11:16.299 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_bytes": 0, 2024-03-24T03:11:16.299 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_items": 0, 2024-03-24T03:11:16.299 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_bytes": 148967, 2024-03-24T03:11:16.299 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_items": 29, 2024-03-24T03:11:16.299 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_bytes": 0, 2024-03-24T03:11:16.299 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_items": 0, 2024-03-24T03:11:16.299 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bytes": 0, 2024-03-24T03:11:16.299 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_items": 0, 2024-03-24T03:11:16.299 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_bytes": 0, 2024-03-24T03:11:16.299 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_items": 0, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_bytes": 0, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_items": 0, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_bytes": 0, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_items": 0, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_bytes": 0, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_items": 0, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_bytes": 0, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_items": 0, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_bytes": 928, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_items": 2, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_bytes": 0, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_items": 0, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_bytes": 0, 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_items": 0 2024-03-24T03:11:16.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 225, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 1406560, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 225, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 225, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 1406560, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:16.302 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:16.302 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:16.302 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:16.302 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:11:16.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:16 smithi019 ceph-mon[35944]: osdmap e103: 12 total, 12 up, 12 in 2024-03-24T03:11:16.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:16 smithi017 ceph-mon[31926]: osdmap e103: 12 total, 12 up, 12 in 2024-03-24T03:11:17.018 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-24T03:11:17.018 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-24T03:11:17.018 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-24T03:11:17.018 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-24T03:11:17.018 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-24T03:11:17.018 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-24T03:11:17.018 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-24T03:11:17.018 DEBUG:tasks.check_counter:Getting stats from g 2024-03-24T03:11:17.019 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-24T03:11:17.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:16 smithi188 ceph-mon[36206]: osdmap e103: 12 total, 12 up, 12 in 2024-03-24T03:11:17.264 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:11:17.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:17 smithi017 ceph-mon[31926]: pgmap v318: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 249 B/s wr, 16 op/s 2024-03-24T03:11:17.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:17 smithi019 ceph-mon[35944]: pgmap v318: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 249 B/s wr, 16 op/s 2024-03-24T03:11:18.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:17 smithi188 ceph-mon[36206]: pgmap v318: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 249 B/s wr, 16 op/s 2024-03-24T03:11:18.771 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:18 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:11:19.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:18 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:11:19.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:18 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T03:11:19.288 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-0": { 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 737, 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 738, 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 247675, 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 234049, 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 6, 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 2, 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.084630137, 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.036541555, 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.017632429, 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.021291606, 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 736, 2024-03-24T03:11:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.039922682, 2024-03-24T03:11:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000054242 2024-03-24T03:11:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-24T03:11:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 0 2024-03-24T03:11:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-1": { 2024-03-24T03:11:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 1589, 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 1578, 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 1815871, 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 557597, 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 4, 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 3, 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.137339736, 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.041322761, 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.058705885, 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.040339726, 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 1577, 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.056672468, 2024-03-24T03:11:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000035936 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 1469904, 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 128240 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-2": { 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 9, 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 164, 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 4740, 2024-03-24T03:11:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 438117, 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 7, 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 5, 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.013190913, 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.008659798, 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.002564291, 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000052762, 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 162, 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.018158933, 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000112092 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 437152 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "cct": { 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "total_workers": 1, 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "unhealthy_workers": 0 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "finisher-MDSRank": { 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "queue_len": 0, 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "complete_latency": { 2024-03-24T03:11:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 3538, 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.086122762, 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000024342 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: "finisher-PurgeQueue": { 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: "queue_len": 0, 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: "complete_latency": { 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": { 2024-03-24T03:11:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: "request": 0, 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "reply": 0, 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "reply_latency": { 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "slow_reply": 0, 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "forward": 0, 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_fetch_complete": 0, 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_fetch_keys": 0, 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_commit": 0, 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_split": 0, 2024-03-24T03:11:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_merge": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_top": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_bottom": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_pin_tail": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_pinned": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_expired": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_with_caps": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "subtrees": 1, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_hit": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_forward": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_discover": 0, 2024-03-24T03:11:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_dir_fetch": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_remote_ino": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_lock": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "load_cent": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "q": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "exported": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "exported_inodes": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "imported": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "imported_inodes": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_dir_fetch": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_backtrace_fetch": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_peer_discover": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rfiles": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rbytes": 0, 2024-03-24T03:11:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rsnaps": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_backtrace_fetch": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_set_tag": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_backtrace_repaired": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_inotable_repaired": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dir_inodes": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dir_base_inodes": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dirfrag_rstats": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_file_inodes": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_inode_file_caps": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_revoke": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_grant": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_trunc": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-24T03:11:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_caps": 0, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_caps_dirty": 0, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_cap_release": 0, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "process_request_cap_release": 0 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache": { 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_update": 0, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_update_receipt": 0, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_try_discover": 0, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_send_discover": 0, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_handle_discover": 0, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays": 0, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays_delayed": 0, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays_enqueuing": 0, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_created": 0, 2024-03-24T03:11:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_enqueued": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_reintegrated": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_migrated": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_processing": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_enqueued": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_prioritized": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_started": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_completed": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_enqueue_scrub": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_exportdir": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_flush": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_fragmentdir": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_fragstats": 0, 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_inodestats": 0 2024-03-24T03:11:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log": { 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "evlrg": 0, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "evadd": 0, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "evex": 0, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "evtrm": 0, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "ev": 7, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "evexg": 0, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "evexd": 0, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "segadd": 0, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "segex": 0, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "segtrm": 0, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "seg": 2, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "segmjr": 2, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "segexg": 0, 2024-03-24T03:11:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "segexd": 0, 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "expos": 4194304, 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "wrpos": 4196347, 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "rdpos": 4196206, 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "jlat": { 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "replayed": 7 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_mem": { 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 3, 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "ino+": 3, 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "ino-": 0, 2024-03-24T03:11:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "dir": 1, 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "dir+": 1, 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "dir-": 0, 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "dn": 0, 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "dn+": 0, 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "dn-": 0, 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "cap": 0, 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "cap+": 0, 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "cap-": 0, 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "rss": 52804, 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "heap": 215420 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_server": { 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "dispatch_client_request": 0, 2024-03-24T03:11:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "dispatch_server_request": 0, 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_request": 0, 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_session": 0, 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_peer_request": 0, 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "req_create_latency": { 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getattr_latency": { 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getfilelock_latency": { 2024-03-24T03:11:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "req_link_latency": { 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookup_latency": { 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookuphash_latency": { 2024-03-24T03:11:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupino_latency": { 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupname_latency": { 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupparent_latency": { 2024-03-24T03:11:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupsnap_latency": { 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lssnap_latency": { 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mkdir_latency": { 2024-03-24T03:11:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.309 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.309 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.309 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mknod_latency": { 2024-03-24T03:11:19.309 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.309 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.309 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mksnap_latency": { 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "req_open_latency": { 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "req_readdir_latency": { 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rename_latency": { 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "req_renamesnap_latency": { 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "req_snapdiff_latency": { 2024-03-24T03:11:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmdir_latency": { 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmsnap_latency": { 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmxattr_latency": { 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setattr_latency": { 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setdirlayout_latency": { 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setfilelock_latency": { 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setlayout_latency": { 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setxattr_latency": { 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "req_symlink_latency": { 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "req_unlink_latency": { 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "cap_revoke_eviction": 0, 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "cap_acquisition_throttle": 0, 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getvxattr_latency": { 2024-03-24T03:11:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_sessions": { 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "session_count": 1, 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "session_add": 3, 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "session_remove": 2, 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "sessions_open": 3, 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "sessions_stale": 0, 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "total_load": 0, 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "average_load": 0, 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "avg_session_uptime": 4111, 2024-03-24T03:11:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "mdthresh_evicted": 0 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool": { 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_bytes": 0, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_items": 0, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_bytes": 0, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_items": 0, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_bytes": 0, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_items": 0, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_items": 0, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_items": 0, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_bytes": 0, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_items": 0, 2024-03-24T03:11:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_items": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_bytes": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_items": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_bytes": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_items": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_items": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_items": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_bytes": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_items": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_bytes": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_items": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-24T03:11:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_items": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_bytes": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_items": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_bytes": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_items": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_bytes": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_items": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_bytes": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_items": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_bytes": 197473, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_items": 55, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_bytes": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_items": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bytes": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_items": 0, 2024-03-24T03:11:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_bytes": 0, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_items": 0, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_bytes": 0, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_items": 0, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_bytes": 10480, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_items": 222, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_bytes": 0, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_items": 0, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_bytes": 0, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_items": 0, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_bytes": 8320, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_items": 13, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_bytes": 0, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_items": 0, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_bytes": 0, 2024-03-24T03:11:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_items": 0 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter": { 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "op_active": 0, 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "op_laggy": 0, 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "op_send": 2089, 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "op_send_bytes": 21, 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "op_resend": 3, 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "op_reply": 2089, 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "op_latency": { 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 2089, 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 5.144940427, 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.002462872 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "op_inflight": 0, 2024-03-24T03:11:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "oplen_avg": { 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 2089, 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 2090 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "op": 2089, 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "op_r": 2089, 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "op_w": 0, 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "op_rmw": 0, 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "op_pg": 0, 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_stat": 1345, 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_create": 0, 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_read": 742, 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_write": 0, 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_writefull": 0, 2024-03-24T03:11:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_writesame": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_append": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_zero": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_truncate": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_delete": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_mapext": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_sparse_read": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_clonerange": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_getxattr": 1, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_setxattr": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_cmpxattr": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_rmxattr": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_resetxattrs": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_call": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_watch": 0, 2024-03-24T03:11:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_notify": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_src_cmpxattr": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_pgls": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_pgls_filter": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_other": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_active": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_send": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_resend": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_ping": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_active": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_send": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_resend": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_active": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_send": 0, 2024-03-24T03:11:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_resend": 0, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_active": 0, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_send": 0, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_resend": 0, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "command_active": 0, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "command_send": 0, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "command_resend": 0, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "map_epoch": 103, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "map_full": 0, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "map_inc": 14, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_sessions": 6, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_session_open": 6, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_session_close": 0, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_laggy": 0, 2024-03-24T03:11:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_wr": 0, 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_rd": 2, 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_del": 0 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "oft": { 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_objs": 0, 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_kv_pairs": 0, 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_updates": 0, 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_removes": 0 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "osd-slow-ops": { 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "slow_ops_count": 0 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "purge_queue": { 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_ops": 0, 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_ops_high_water": 0, 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing": 0, 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_high_water": 0, 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executed_ops": 0, 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executed": 0, 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_item_in_journal": 0 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 2331, 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 1843188, 2024-03-24T03:11:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 2331, 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 2331, 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 1843188, 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-objecter_bytes": { 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-03-24T03:11:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 2089, 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 2048, 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 2089, 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 2048, 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-objecter_ops": { 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 1024, 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 2089, 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 2089, 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 2089, 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 2089, 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-03-24T03:11:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-write_buf_throttle": { 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 3758096384, 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2024-03-24T03:11:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 0, 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 0, 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-write_buf_throttle-0x55bb64f4e5a0": { 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 3758096384, 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2024-03-24T03:11:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2024-03-24T03:11:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2024-03-24T03:11:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 0, 2024-03-24T03:11:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 0, 2024-03-24T03:11:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-03-24T03:11:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:19.333 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:11:19.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:19 smithi017 ceph-mon[31926]: pgmap v319: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-03-24T03:11:19.994 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-24T03:11:19.994 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-24T03:11:19.994 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-03-24T03:11:19.994 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-03-24T03:11:19.994 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-03-24T03:11:19.994 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-03-24T03:11:19.994 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-03-24T03:11:19.994 DEBUG:tasks.check_counter:Getting stats from j 2024-03-24T03:11:19.995 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-24T03:11:20.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:19 smithi188 ceph-mon[36206]: pgmap v319: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-03-24T03:11:20.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:19 smithi019 ceph-mon[35944]: pgmap v319: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-03-24T03:11:20.780 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T03:11:21.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:21.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:21.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: pgmap v320: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:11:21.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:21.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:21.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-03-24T03:11:21.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-24T03:11:21.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi019 to 175.2M 2024-03-24T03:11:21.457 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:21 smithi017 ceph-mon[31926]: Unable to set osd_memory_target on smithi019 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-03-24T03:11:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: pgmap v320: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:11:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-03-24T03:11:21.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-24T03:11:21.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi019 to 175.2M 2024-03-24T03:11:21.549 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:21 smithi188 ceph-mon[36206]: Unable to set osd_memory_target on smithi019 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-03-24T03:11:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: pgmap v320: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:11:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:21.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-03-24T03:11:21.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-24T03:11:21.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:21.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi019 to 175.2M 2024-03-24T03:11:21.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:21 smithi019 ceph-mon[35944]: Unable to set osd_memory_target on smithi019 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-03-24T03:11:22.623 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-24T03:11:22.623 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-0": { 2024-03-24T03:11:22.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 0, 2024-03-24T03:11:22.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 2, 2024-03-24T03:11:22.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 0, 2024-03-24T03:11:22.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 0, 2024-03-24T03:11:22.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 4, 2024-03-24T03:11:22.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 0, 2024-03-24T03:11:22.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.001284706, 2024-03-24T03:11:22.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.000000000, 2024-03-24T03:11:22.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.000402792, 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 0 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:22.624 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-1": { 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 6, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 166, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 3392, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 184624, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 5, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 3, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.015480234, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.010612992, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.002257147, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 164, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.011422432, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000069648 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:22.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 184624 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-2": { 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 225, 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 215, 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 1438528, 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 128688, 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 2, 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 1, 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.043849119, 2024-03-24T03:11:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.013498561, 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.034671494, 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.004487107, 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 214, 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.010916418, 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000051011 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 1438528, 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 128688 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "cct": { 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "total_workers": 0, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "unhealthy_workers": 0 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool": { 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_bytes": 0, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_items": 0, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_bytes": 0, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_items": 0, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_bytes": 0, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_items": 0, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_items": 0, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_items": 0, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_bytes": 0, 2024-03-24T03:11:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_items": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_items": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_bytes": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_items": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_bytes": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_items": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_items": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_items": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_bytes": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_items": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_bytes": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_items": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_items": 0, 2024-03-24T03:11:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_bytes": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_items": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_bytes": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_items": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_bytes": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_items": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_bytes": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_items": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_bytes": 149752, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_items": 31, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_bytes": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_items": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bytes": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_items": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_bytes": 0, 2024-03-24T03:11:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_items": 0, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_bytes": 0, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_items": 0, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_bytes": 0, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_items": 0, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_bytes": 0, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_items": 0, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_bytes": 0, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_items": 0, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_bytes": 928, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_items": 2, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_bytes": 0, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_items": 0, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_bytes": 0, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_items": 0 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-24T03:11:22.631 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 227, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 1406820, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 227, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 227, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 1406820, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:22.632 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-24T03:11:22.633 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-24T03:11:23.292 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-24T03:11:23.292 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-24T03:11:23.292 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-24T03:11:23.292 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-24T03:11:23.292 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-24T03:11:23.292 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-24T03:11:23.292 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-24T03:11:23.292 DEBUG:tasks.check_counter:Getting stats from b 2024-03-24T03:11:23.293 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-24T03:11:23.556 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T03:11:23.677 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:23 smithi019 ceph-mon[35944]: pgmap v321: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:11:23.705 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:23 smithi017 ceph-mon[31926]: pgmap v321: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:11:23.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:23 smithi188 ceph-mon[36206]: pgmap v321: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:11:24.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:24 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:24.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:24 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:24.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:24 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:24.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:24 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:24.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:24 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:24.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:24 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:24.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:24 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:11:24.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:24 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:11:24.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:24 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:24.817 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:24 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:24.817 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:24 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:24.817 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:24 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:24.817 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:24 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:24.818 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:24 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:24.818 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:24 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:24.818 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:24 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:11:24.818 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:24 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:11:24.818 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:24 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:24 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:24 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:24 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:24 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:24 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:24 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-24T03:11:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:24 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T03:11:24.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:24 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T03:11:24.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:24 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' 2024-03-24T03:11:25.427 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-24T03:11:25.427 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-03-24T03:11:25.427 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 297, 2024-03-24T03:11:25.427 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 404, 2024-03-24T03:11:25.427 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1501551, 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 494585, 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 13, 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.045873858, 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.012815163, 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.038571273, 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.004118263, 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 402, 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.027371266, 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000068087 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:25.429 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4, 2024-03-24T03:11:25.429 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000388, 2024-03-24T03:11:25.429 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000097 2024-03-24T03:11:25.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1497360, 2024-03-24T03:11:25.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 442864 2024-03-24T03:11:25.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.429 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-03-24T03:11:25.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 306, 2024-03-24T03:11:25.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 546, 2024-03-24T03:11:25.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 91465, 2024-03-24T03:11:25.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 415331, 2024-03-24T03:11:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 8, 2024-03-24T03:11:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-03-24T03:11:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.037617211, 2024-03-24T03:11:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.014820931, 2024-03-24T03:11:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.017669540, 2024-03-24T03:11:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.002982117, 2024-03-24T03:11:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 545, 2024-03-24T03:11:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.028831802, 2024-03-24T03:11:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000052902 2024-03-24T03:11:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:25.431 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 767, 2024-03-24T03:11:25.431 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000146872, 2024-03-24T03:11:25.431 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000191 2024-03-24T03:11:25.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-03-24T03:11:25.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 132928 2024-03-24T03:11:25.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.431 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-03-24T03:11:25.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 30, 2024-03-24T03:11:25.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 60, 2024-03-24T03:11:25.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 7897, 2024-03-24T03:11:25.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 24090, 2024-03-24T03:11:25.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-03-24T03:11:25.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-03-24T03:11:25.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.004843965, 2024-03-24T03:11:25.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.001146946, 2024-03-24T03:11:25.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.003107011, 2024-03-24T03:11:25.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000427795, 2024-03-24T03:11:25.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:25.432 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 58, 2024-03-24T03:11:25.432 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.015408917, 2024-03-24T03:11:25.432 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000265670 2024-03-24T03:11:25.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 56, 2024-03-24T03:11:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000008522, 2024-03-24T03:11:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000152 2024-03-24T03:11:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3488, 2024-03-24T03:11:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 11056 2024-03-24T03:11:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-03-24T03:11:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-03-24T03:11:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-03-24T03:11:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.434 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-03-24T03:11:25.434 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-03-24T03:11:25.434 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-03-24T03:11:25.434 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 37, 2024-03-24T03:11:25.434 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.002004882, 2024-03-24T03:11:25.434 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000054186 2024-03-24T03:11:25.434 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:25.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.434 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-03-24T03:11:25.434 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-03-24T03:11:25.434 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2, 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000006969, 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000003484 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 7, 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 4, 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4, 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000657795, 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000164448 2024-03-24T03:11:25.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 3, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 11, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 14, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 1, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 13, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 13, 2024-03-24T03:11:25.436 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 2, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 2, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 2, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 12, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 9, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 2, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-03-24T03:11:25.437 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 4, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 234, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-03-24T03:11:25.438 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-03-24T03:11:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-03-24T03:11:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-03-24T03:11:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-03-24T03:11:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-03-24T03:11:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-03-24T03:11:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-03-24T03:11:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-03-24T03:11:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-03-24T03:11:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-24T03:11:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-24T03:11:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-03-24T03:11:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-03-24T03:11:25.440 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-03-24T03:11:25.440 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-03-24T03:11:25.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.440 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-03-24T03:11:25.440 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-03-24T03:11:25.440 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 8, 2024-03-24T03:11:25.440 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 5, 2024-03-24T03:11:25.440 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 6, 2024-03-24T03:11:25.440 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-03-24T03:11:25.440 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-03-24T03:11:25.441 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-03-24T03:11:25.441 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-03-24T03:11:25.441 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-03-24T03:11:25.441 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-03-24T03:11:25.441 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-03-24T03:11:25.441 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-03-24T03:11:25.441 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-03-24T03:11:25.441 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-03-24T03:11:25.441 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-03-24T03:11:25.441 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-03-24T03:11:25.441 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-03-24T03:11:25.441 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 4, 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 4, 2024-03-24T03:11:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 2, 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4195591, 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 0, 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-03-24T03:11:25.443 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3, 2024-03-24T03:11:25.444 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.004653929, 2024-03-24T03:11:25.444 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001551309 2024-03-24T03:11:25.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.444 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 0 2024-03-24T03:11:25.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.444 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-03-24T03:11:25.444 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 17, 2024-03-24T03:11:25.444 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 17, 2024-03-24T03:11:25.444 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-03-24T03:11:25.444 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 16, 2024-03-24T03:11:25.444 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 17, 2024-03-24T03:11:25.444 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 1, 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 14, 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 14, 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 2, 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 2, 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 49640, 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 223612 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 7, 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 7, 2024-03-24T03:11:25.445 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 17, 2024-03-24T03:11:25.446 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-03-24T03:11:25.446 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-03-24T03:11:25.446 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.446 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.446 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.446 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-03-24T03:11:25.446 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3, 2024-03-24T03:11:25.446 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000561281, 2024-03-24T03:11:25.446 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000187093 2024-03-24T03:11:25.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.446 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.447 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000096514, 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000096514 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.448 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.449 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-03-24T03:11:25.450 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.450 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.450 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.450 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-03-24T03:11:25.450 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.450 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.450 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.451 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-03-24T03:11:25.451 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.451 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.451 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.451 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-03-24T03:11:25.451 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.451 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.451 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-03-24T03:11:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-03-24T03:11:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-03-24T03:11:25.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.454 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.454 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-03-24T03:11:25.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.454 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.454 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-03-24T03:11:25.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.454 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.455 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-03-24T03:11:25.457 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 2, 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 2, 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 0, 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 2, 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 3, 2024-03-24T03:11:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 1, 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 587, 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-24T03:11:25.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-03-24T03:11:25.460 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-03-24T03:11:25.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 271925, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 98, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 176, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 2, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 10240, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 216, 2024-03-24T03:11:25.462 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 68688, 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 257, 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-03-24T03:11:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 40, 2024-03-24T03:11:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 11800, 2024-03-24T03:11:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 33, 2024-03-24T03:11:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 40, 2024-03-24T03:11:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-03-24T03:11:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 40, 2024-03-24T03:11:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.231997972, 2024-03-24T03:11:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.005799949 2024-03-24T03:11:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-03-24T03:11:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-03-24T03:11:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 40, 2024-03-24T03:11:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 63 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 40, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 0, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 40, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 0, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 10, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 0, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 3, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 7, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-03-24T03:11:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 6, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 0, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 20, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-03-24T03:11:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-03-24T03:11:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 103, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 14, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 9, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 9, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 16, 2024-03-24T03:11:25.468 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 0, 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 1 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 1, 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-03-24T03:11:25.469 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-03-24T03:11:25.470 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-03-24T03:11:25.470 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-03-24T03:11:25.470 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-03-24T03:11:25.470 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-03-24T03:11:25.470 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-03-24T03:11:25.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.470 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-24T03:11:25.470 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:25.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-03-24T03:11:25.470 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:25.470 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 629, 2024-03-24T03:11:25.470 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1526727, 2024-03-24T03:11:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 629, 2024-03-24T03:11:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-03-24T03:11:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-03-24T03:11:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 629, 2024-03-24T03:11:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1526727, 2024-03-24T03:11:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 40, 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 11800, 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 40, 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 11800, 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 40, 2024-03-24T03:11:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 40, 2024-03-24T03:11:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 40, 2024-03-24T03:11:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 40, 2024-03-24T03:11:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-03-24T03:11:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-03-24T03:11:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.475 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x5614fc5b8500": { 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 4, 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1287, 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 4, 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3, 2024-03-24T03:11:25.476 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1287, 2024-03-24T03:11:25.477 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:25.477 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:25.477 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:25.477 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:25.477 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:25.477 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:25.477 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-24T03:11:25.735 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:25 smithi019 ceph-mon[35944]: Adjusting osd_memory_target on smithi017 to 177.2M 2024-03-24T03:11:25.735 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:25 smithi019 ceph-mon[35944]: Unable to set osd_memory_target on smithi017 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-24T03:11:25.735 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:25 smithi019 ceph-mon[35944]: pgmap v322: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:11:25.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:25 smithi188 ceph-mon[36206]: Adjusting osd_memory_target on smithi017 to 177.2M 2024-03-24T03:11:25.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:25 smithi188 ceph-mon[36206]: Unable to set osd_memory_target on smithi017 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-24T03:11:25.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:25 smithi188 ceph-mon[36206]: pgmap v322: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:11:25.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:25 smithi017 ceph-mon[31926]: Adjusting osd_memory_target on smithi017 to 177.2M 2024-03-24T03:11:25.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:25 smithi017 ceph-mon[31926]: Unable to set osd_memory_target on smithi017 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-24T03:11:25.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:25 smithi017 ceph-mon[31926]: pgmap v322: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-24T03:11:26.242 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-24T03:11:26.243 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-24T03:11:26.243 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=0 2024-03-24T03:11:26.243 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=8 2024-03-24T03:11:26.243 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=0 2024-03-24T03:11:26.243 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-03-24T03:11:26.243 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-03-24T03:11:26.243 DEBUG:tasks.check_counter:Getting stats from e 2024-03-24T03:11:26.243 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-24T03:11:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T03:11:27.796 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:27 smithi019 ceph-mon[35944]: pgmap v323: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-24T03:11:27.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:27 smithi188 ceph-mon[36206]: pgmap v323: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-24T03:11:27.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:27 smithi017 ceph-mon[31926]: pgmap v323: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-24T03:11:28.311 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-24T03:11:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-03-24T03:11:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1297, 2024-03-24T03:11:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1283, 2024-03-24T03:11:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 5998682, 2024-03-24T03:11:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 465224, 2024-03-24T03:11:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-03-24T03:11:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-03-24T03:11:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.117330120, 2024-03-24T03:11:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.036193029, 2024-03-24T03:11:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.056887858, 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.028682290, 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1283, 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.035700655, 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000027825 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1474768, 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 132592 2024-03-24T03:11:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 984, 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1146, 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 3554298, 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 753126, 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.061707498, 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.018343818, 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.017722712, 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.020263486, 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:28.313 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1143, 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.031901334, 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000027910 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 440784 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-03-24T03:11:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 296, 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 298, 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 4267754, 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 94135, 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.025011293, 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.005722797, 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.008842458, 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.008445249, 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 296, 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.007730330, 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000026115 2024-03-24T03:11:28.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-03-24T03:11:28.316 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3873, 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.081090044, 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000020937 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.317 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-03-24T03:11:28.318 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 27, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 27, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 27, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 2, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-03-24T03:11:28.319 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 4, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 234, 2024-03-24T03:11:28.320 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 6, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-24T03:11:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-03-24T03:11:28.322 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-03-24T03:11:28.323 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 4411, 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-03-24T03:11:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 7, 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 15886966, 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 15886830, 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 4411 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-03-24T03:11:28.325 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 18, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 21, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 3, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 12, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 12, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 27, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 27, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 50820, 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215420 2024-03-24T03:11:28.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.327 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.328 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.329 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-03-24T03:11:28.330 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.331 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.331 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.331 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-03-24T03:11:28.331 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.331 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.331 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.332 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.333 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.334 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-03-24T03:11:28.335 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.337 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 2, 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 7, 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 5, 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 5, 2024-03-24T03:11:28.338 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 4209, 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-24T03:11:28.339 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-03-24T03:11:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-24T03:11:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-03-24T03:11:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-03-24T03:11:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-03-24T03:11:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-24T03:11:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-03-24T03:11:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-03-24T03:11:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-03-24T03:11:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-03-24T03:11:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-03-24T03:11:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-24T03:11:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-03-24T03:11:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 189672, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 59, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 10240, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 216, 2024-03-24T03:11:28.342 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 87386, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 602, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 2324, 2024-03-24T03:11:28.343 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 26, 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 4, 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 2324, 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2324, 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 4.387960879, 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001888107 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2324, 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2325 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 2324, 2024-03-24T03:11:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 2324, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 1490, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 831, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-03-24T03:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 2, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-03-24T03:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 103, 2024-03-24T03:11:28.347 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 14, 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 5, 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 5, 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-03-24T03:11:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-24T03:11:28.349 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 2573, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 13573498, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 2573, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2573, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 13573498, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.350 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2324, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 11692672, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2324, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 11692672, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.351 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2324, 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 2324, 2024-03-24T03:11:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2324, 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2324, 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:28.353 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x555a8aeae5a0": { 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:28.355 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-24T03:11:29.143 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-24T03:11:29.144 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-24T03:11:29.144 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-24T03:11:29.144 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-03-24T03:11:29.144 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-03-24T03:11:29.144 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-24T03:11:29.144 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-24T03:11:29.144 DEBUG:tasks.check_counter:Getting stats from h 2024-03-24T03:11:29.145 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-24T03:11:29.422 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T03:11:29.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:29 smithi188 ceph-mon[36206]: pgmap v324: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:29.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:29 smithi019 ceph-mon[35944]: pgmap v324: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:29.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:29 smithi017 ceph-mon[31926]: pgmap v324: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:31.210 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-24T03:11:31.210 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-03-24T03:11:31.210 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 287, 2024-03-24T03:11:31.210 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 481, 2024-03-24T03:11:31.210 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 84944, 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 204191, 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 9, 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.032335045, 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.012230337, 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.013903814, 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.002945481, 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 480, 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.019313013, 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000040235 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.211 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 717, 2024-03-24T03:11:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000139581, 2024-03-24T03:11:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000194 2024-03-24T03:11:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 4640, 2024-03-24T03:11:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 5600 2024-03-24T03:11:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-03-24T03:11:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 293, 2024-03-24T03:11:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 281, 2024-03-24T03:11:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1487369, 2024-03-24T03:11:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 191289, 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 10, 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 6, 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.039278790, 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.010132738, 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.035372480, 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.004242953, 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 280, 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.017082889, 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000061010 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 12, 2024-03-24T03:11:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000001465, 2024-03-24T03:11:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000122 2024-03-24T03:11:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1484288, 2024-03-24T03:11:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 137136 2024-03-24T03:11:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-03-24T03:11:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 27, 2024-03-24T03:11:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 190, 2024-03-24T03:11:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 7161, 2024-03-24T03:11:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 453462, 2024-03-24T03:11:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 9, 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 6, 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.011966148, 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.005355941, 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.004793606, 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000405843, 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 187, 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.020021832, 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000107068 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 16, 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000003376, 2024-03-24T03:11:31.215 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000211 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 2496, 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 445232 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 48, 2024-03-24T03:11:31.216 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.002931091, 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000061064 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2, 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000007960, 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000003980 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 4, 2024-03-24T03:11:31.217 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 3, 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3, 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.001344779, 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000448259 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 1, 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 11, 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-03-24T03:11:31.218 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 13, 2024-03-24T03:11:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-03-24T03:11:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 13, 2024-03-24T03:11:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-03-24T03:11:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 12, 2024-03-24T03:11:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 1, 2024-03-24T03:11:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-03-24T03:11:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-03-24T03:11:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 2, 2024-03-24T03:11:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 10, 2024-03-24T03:11:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 5, 2024-03-24T03:11:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 1, 2024-03-24T03:11:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 2, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 4, 2024-03-24T03:11:31.220 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 234, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-03-24T03:11:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 3, 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 2, 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 4, 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-03-24T03:11:31.222 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-03-24T03:11:31.223 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 7, 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 7, 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 2, 2024-03-24T03:11:31.224 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4196347, 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 0, 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 6, 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.199904897, 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.033317482 2024-03-24T03:11:31.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 0 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 16, 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 16, 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 15, 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 16, 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 1, 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 13, 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 14, 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 1, 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-03-24T03:11:31.226 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 2, 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 2, 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 52060, 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 223612 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 5, 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 4, 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 16, 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.227 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000426347, 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000426347 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2, 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000918432, 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000459216 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-03-24T03:11:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.230 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.231 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.232 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-03-24T03:11:31.232 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.232 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.232 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.232 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-03-24T03:11:31.232 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-03-24T03:11:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-03-24T03:11:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-03-24T03:11:31.237 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.238 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 3, 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 2, 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 2, 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 1, 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 568, 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.239 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-03-24T03:11:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-03-24T03:11:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 247377, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 87, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-03-24T03:11:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 10240, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 216, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 63880, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 247, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-03-24T03:11:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 45, 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 12580, 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 33, 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 45, 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 45, 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.423996221, 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.009422138 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-03-24T03:11:31.244 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 45, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 69 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 45, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 0, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 45, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 0, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 10, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 0, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 6, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 8, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-03-24T03:11:31.245 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 6, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 0, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 20, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-03-24T03:11:31.246 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-03-24T03:11:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 103, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 14, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 9, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 9, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 17, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 0, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 2 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.248 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 1, 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-03-24T03:11:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 603, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1507297, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 603, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 603, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1507297, 2024-03-24T03:11:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 45, 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 12580, 2024-03-24T03:11:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 45, 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 12580, 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:31.252 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 45, 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 45, 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 45, 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 45, 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55a3bdb00500": { 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 7, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 2043, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 7, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 6, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2043, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:31.255 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:31.256 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:31.256 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:31.256 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:31.256 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-24T03:11:31.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:31 smithi188 ceph-mon[36206]: pgmap v325: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:31.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:31 smithi019 ceph-mon[35944]: pgmap v325: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:31.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:31 smithi017 ceph-mon[31926]: pgmap v325: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:32.029 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-24T03:11:32.030 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-24T03:11:32.030 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-24T03:11:32.030 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=3 2024-03-24T03:11:32.030 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-03-24T03:11:32.030 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-24T03:11:32.030 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-24T03:11:32.030 DEBUG:tasks.check_counter:Getting stats from k 2024-03-24T03:11:32.031 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-24T03:11:32.337 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T03:11:33.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:33 smithi188 ceph-mon[36206]: pgmap v326: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:33.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:33 smithi019 ceph-mon[35944]: pgmap v326: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:33.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:33 smithi017 ceph-mon[31926]: pgmap v326: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:34.097 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-24T03:11:34.097 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-03-24T03:11:34.097 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 249, 2024-03-24T03:11:34.097 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 238, 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1473186, 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 134180, 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.036361379, 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.008854808, 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.032977552, 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.003920622, 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 237, 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.007917846, 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000033408 2024-03-24T03:11:34.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1472352, 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 133856 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 697, 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 861, 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 171312, 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 662151, 2024-03-24T03:11:34.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.057605756, 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.018725543, 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.013050251, 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.019888301, 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 858, 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.026776678, 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000031208 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 441136 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1520, 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1520, 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 458215, 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 482594, 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.100423594, 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.033518668, 2024-03-24T03:11:34.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.026075248, 2024-03-24T03:11:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.032218266, 2024-03-24T03:11:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1519, 2024-03-24T03:11:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.035971331, 2024-03-24T03:11:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000023680 2024-03-24T03:11:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3684, 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.074370507, 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000020187 2024-03-24T03:11:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-03-24T03:11:34.104 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 0, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 0, 2024-03-24T03:11:34.105 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 1, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-03-24T03:11:34.106 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-03-24T03:11:34.107 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-03-24T03:11:34.108 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-03-24T03:11:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 8, 2024-03-24T03:11:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4196488, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 4196347, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.111 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 8 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 3, 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 3, 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 1, 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 1, 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 0, 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 0, 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-03-24T03:11:34.112 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 47792, 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215420 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.114 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.115 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.117 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-03-24T03:11:34.117 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.117 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.117 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.117 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-03-24T03:11:34.117 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.117 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.117 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.118 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-03-24T03:11:34.118 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.118 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.118 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.118 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-03-24T03:11:34.118 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.118 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.118 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.118 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.123 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 0, 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 3, 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 3, 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 2, 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-03-24T03:11:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 3166, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-24T03:11:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-03-24T03:11:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 190179, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 53, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-03-24T03:11:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 10240, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 216, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 8320, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 13, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-03-24T03:11:34.128 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 2212, 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 2212, 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2212, 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2.072981689, 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000937152 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-03-24T03:11:34.129 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2212, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2213 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 2212, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 2212, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 1447, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 763, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-03-24T03:11:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-03-24T03:11:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-03-24T03:11:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 103, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 14, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 5, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 5, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-03-24T03:11:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-03-24T03:11:34.134 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 2462, 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1865357, 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 2462, 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2462, 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1865357, 2024-03-24T03:11:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2212, 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 2189, 2024-03-24T03:11:34.136 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2212, 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2189, 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2212, 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 2212, 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2212, 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2212, 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:34.138 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x555c304e05a0": { 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-03-24T03:11:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-24T03:11:34.140 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-24T03:11:34.881 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-24T03:11:34.881 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-24T03:11:34.881 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-03-24T03:11:34.881 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-03-24T03:11:34.881 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-03-24T03:11:34.881 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-24T03:11:34.881 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-24T03:11:34.881 DEBUG:tasks.check_counter:Getting stats from c 2024-03-24T03:11:34.882 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-24T03:11:35.186 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T03:11:35.582 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:35 smithi188 ceph-mon[36206]: pgmap v327: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:35.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:35 smithi019 ceph-mon[35944]: pgmap v327: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:35.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:35 smithi017 ceph-mon[31926]: pgmap v327: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:37.024 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-03-24T03:11:37.025 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-03-24T03:11:37.025 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 308, 2024-03-24T03:11:37.025 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 487, 2024-03-24T03:11:37.025 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 91575, 2024-03-24T03:11:37.025 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 165017, 2024-03-24T03:11:37.025 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 8, 2024-03-24T03:11:37.025 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-03-24T03:11:37.025 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.036837162, 2024-03-24T03:11:37.025 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.016709984, 2024-03-24T03:11:37.025 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.014344815, 2024-03-24T03:11:37.026 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.003366445, 2024-03-24T03:11:37.026 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:37.026 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 489, 2024-03-24T03:11:37.026 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.026424227, 2024-03-24T03:11:37.026 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000054037 2024-03-24T03:11:37.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.026 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:37.026 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 730, 2024-03-24T03:11:37.026 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000139830, 2024-03-24T03:11:37.026 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000191 2024-03-24T03:11:37.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.026 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 300, 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 410, 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 1499630, 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 463272, 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 6, 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 5, 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.041376333, 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.013177319, 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.034404317, 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.004200183, 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:37.027 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 409, 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.032170523, 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000078656 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 1495840, 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 451168 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-03-24T03:11:37.028 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 24, 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 66, 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 7666, 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 180666, 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 14, 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.007440639, 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.001559379, 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.004442484, 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000115419, 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 56, 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.006195123, 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000110627 2024-03-24T03:11:37.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 23, 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000003903, 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000169 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 5696, 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 133808 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.030 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 65, 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.001310033, 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000020154 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2, 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000015898, 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000007949 2024-03-24T03:11:37.031 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 0, 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 0, 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 0, 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 11, 2024-03-24T03:11:37.032 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 14, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 1, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 13, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 12, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 2, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 5, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 3, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-03-24T03:11:37.033 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 2, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-03-24T03:11:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 4, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 234, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-03-24T03:11:37.035 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 8, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 5, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 6, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-03-24T03:11:37.036 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-03-24T03:11:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 8, 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 8, 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 2, 2024-03-24T03:11:37.038 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 2, 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 2, 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 4194304, 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 4196488, 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 7, 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.022194398, 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003170628 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-03-24T03:11:37.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 17, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 17, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 0, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 16, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 17, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 1, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 14, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 14, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 0, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 0, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 0, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 52044, 2024-03-24T03:11:37.040 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 223612 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 0, 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 0, 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 10, 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.041 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-03-24T03:11:37.042 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.043 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.044 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-03-24T03:11:37.045 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.045 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.045 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.045 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-03-24T03:11:37.045 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.045 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.045 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.046 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-03-24T03:11:37.046 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.046 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.046 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.046 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-03-24T03:11:37.046 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.046 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.046 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.047 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.050 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.051 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 0, 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 3, 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 3, 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 0, 2024-03-24T03:11:37.052 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 0, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 0, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 300, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-03-24T03:11:37.053 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-24T03:11:37.054 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-03-24T03:11:37.055 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 229831, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 67, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 10240, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 216, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-03-24T03:11:37.056 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 67944, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 252, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 45, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 12666, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 33, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 45, 2024-03-24T03:11:37.057 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 45, 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.332994411, 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.007399875 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 45, 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 67 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 45, 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 0, 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 45, 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-03-24T03:11:37.058 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 0, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 10, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 0, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 7, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 9, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 6, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 0, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 20, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-03-24T03:11:37.059 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-03-24T03:11:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 103, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 14, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 8, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 8, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-03-24T03:11:37.061 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 14, 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 0, 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 1 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-03-24T03:11:37.062 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 628, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 1524458, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:37.063 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 628, 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 628, 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1524458, 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:37.064 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 45, 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 12666, 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 45, 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 12666, 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.065 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 45, 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 45, 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 45, 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 45, 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.066 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-03-24T03:11:37.067 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x5639f1524500": { 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 8, 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 2184, 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 8, 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-03-24T03:11:37.068 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-03-24T03:11:37.069 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 7, 2024-03-24T03:11:37.069 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 2184, 2024-03-24T03:11:37.069 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:37.069 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:37.069 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:37.069 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:37.069 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:37.069 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:37.069 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-03-24T03:11:37.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:37 smithi188 ceph-mon[36206]: pgmap v328: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:37.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:37 smithi019 ceph-mon[35944]: pgmap v328: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:37.860 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-24T03:11:37.860 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-24T03:11:37.860 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-03-24T03:11:37.860 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=8 2024-03-24T03:11:37.860 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-03-24T03:11:37.860 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-24T03:11:37.860 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-24T03:11:37.860 DEBUG:tasks.check_counter:Getting stats from f 2024-03-24T03:11:37.861 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-24T03:11:37.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:37 smithi017 ceph-mon[31926]: pgmap v328: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:38.131 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T03:11:39.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:39 smithi188 ceph-mon[36206]: pgmap v329: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:39.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:39 smithi019 ceph-mon[35944]: pgmap v329: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:39.895 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-03-24T03:11:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-03-24T03:11:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 307, 2024-03-24T03:11:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 478, 2024-03-24T03:11:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 91920, 2024-03-24T03:11:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 215950, 2024-03-24T03:11:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 8, 2024-03-24T03:11:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-03-24T03:11:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.036105463, 2024-03-24T03:11:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.016415922, 2024-03-24T03:11:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.013822619, 2024-03-24T03:11:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.003436018, 2024-03-24T03:11:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 480, 2024-03-24T03:11:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.030274981, 2024-03-24T03:11:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000063072 2024-03-24T03:11:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 722, 2024-03-24T03:11:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000136160, 2024-03-24T03:11:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000188 2024-03-24T03:11:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-24T03:11:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-03-24T03:11:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-03-24T03:11:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 30, 2024-03-24T03:11:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 96, 2024-03-24T03:11:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 6612, 2024-03-24T03:11:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 146723, 2024-03-24T03:11:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 9, 2024-03-24T03:11:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-03-24T03:11:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.008934932, 2024-03-24T03:11:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.003147795, 2024-03-24T03:11:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.004353212, 2024-03-24T03:11:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000248506, 2024-03-24T03:11:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 93, 2024-03-24T03:11:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.011131963, 2024-03-24T03:11:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000119698 2024-03-24T03:11:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 53, 2024-03-24T03:11:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000014691, 2024-03-24T03:11:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000277 2024-03-24T03:11:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 2000, 2024-03-24T03:11:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 134272 2024-03-24T03:11:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-03-24T03:11:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 300, 2024-03-24T03:11:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 414, 2024-03-24T03:11:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 1496660, 2024-03-24T03:11:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 504890, 2024-03-24T03:11:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 11, 2024-03-24T03:11:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 5, 2024-03-24T03:11:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.043248323, 2024-03-24T03:11:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.013150397, 2024-03-24T03:11:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.036152680, 2024-03-24T03:11:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.004163867, 2024-03-24T03:11:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 410, 2024-03-24T03:11:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.024991241, 2024-03-24T03:11:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000060954 2024-03-24T03:11:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.902 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:39.902 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 11, 2024-03-24T03:11:39.902 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000001555, 2024-03-24T03:11:39.902 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000141 2024-03-24T03:11:39.902 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.902 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 1494480, 2024-03-24T03:11:39.902 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 451968 2024-03-24T03:11:39.902 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.903 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-03-24T03:11:39.903 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-03-24T03:11:39.903 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-03-24T03:11:39.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.903 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-03-24T03:11:39.903 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-03-24T03:11:39.903 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-03-24T03:11:39.903 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 46, 2024-03-24T03:11:39.903 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.001914683, 2024-03-24T03:11:39.903 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000041623 2024-03-24T03:11:39.904 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:39.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.904 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-03-24T03:11:39.904 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-03-24T03:11:39.904 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-03-24T03:11:39.904 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2, 2024-03-24T03:11:39.904 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000005871, 2024-03-24T03:11:39.904 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000002935 2024-03-24T03:11:39.904 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:39.905 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.905 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-03-24T03:11:39.905 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 4, 2024-03-24T03:11:39.905 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 3, 2024-03-24T03:11:39.905 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-03-24T03:11:39.905 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3, 2024-03-24T03:11:39.905 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000492469, 2024-03-24T03:11:39.905 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000164156 2024-03-24T03:11:39.905 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.905 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-03-24T03:11:39.906 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 1, 2024-03-24T03:11:39.906 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-03-24T03:11:39.906 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-03-24T03:11:39.906 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 11, 2024-03-24T03:11:39.906 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-03-24T03:11:39.906 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-03-24T03:11:39.906 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 14, 2024-03-24T03:11:39.906 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 1, 2024-03-24T03:11:39.906 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 13, 2024-03-24T03:11:39.907 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-03-24T03:11:39.907 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 12, 2024-03-24T03:11:39.907 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-03-24T03:11:39.907 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 1, 2024-03-24T03:11:39.907 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 1, 2024-03-24T03:11:39.907 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 2, 2024-03-24T03:11:39.907 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 9, 2024-03-24T03:11:39.907 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 6, 2024-03-24T03:11:39.907 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 1, 2024-03-24T03:11:39.907 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 2, 2024-03-24T03:11:39.907 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-03-24T03:11:39.908 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-03-24T03:11:39.908 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-03-24T03:11:39.908 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-03-24T03:11:39.908 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-03-24T03:11:39.908 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-03-24T03:11:39.908 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-03-24T03:11:39.908 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-03-24T03:11:39.908 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-03-24T03:11:39.908 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-03-24T03:11:39.908 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-03-24T03:11:39.909 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-03-24T03:11:39.909 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 4, 2024-03-24T03:11:39.909 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 234, 2024-03-24T03:11:39.909 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-03-24T03:11:39.909 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-03-24T03:11:39.909 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-03-24T03:11:39.909 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-03-24T03:11:39.909 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-03-24T03:11:39.909 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-03-24T03:11:39.909 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-03-24T03:11:39.909 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-03-24T03:11:39.910 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-03-24T03:11:39.910 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-03-24T03:11:39.910 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-03-24T03:11:39.910 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-03-24T03:11:39.910 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-03-24T03:11:39.910 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-24T03:11:39.910 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-24T03:11:39.910 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-03-24T03:11:39.910 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-03-24T03:11:39.910 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-03-24T03:11:39.911 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-03-24T03:11:39.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.911 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-03-24T03:11:39.911 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-03-24T03:11:39.911 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 6, 2024-03-24T03:11:39.911 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 4, 2024-03-24T03:11:39.911 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 5, 2024-03-24T03:11:39.911 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-03-24T03:11:39.911 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-03-24T03:11:39.911 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-03-24T03:11:39.912 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-03-24T03:11:39.912 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-03-24T03:11:39.912 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-03-24T03:11:39.912 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-03-24T03:11:39.912 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-03-24T03:11:39.912 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-03-24T03:11:39.912 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-03-24T03:11:39.912 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-03-24T03:11:39.912 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-03-24T03:11:39.912 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-03-24T03:11:39.913 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-03-24T03:11:39.913 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-03-24T03:11:39.913 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-03-24T03:11:39.913 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-03-24T03:11:39.913 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-03-24T03:11:39.913 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-03-24T03:11:39.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.913 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-03-24T03:11:39.913 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-03-24T03:11:39.913 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 7, 2024-03-24T03:11:39.914 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-03-24T03:11:39.914 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-03-24T03:11:39.914 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 7, 2024-03-24T03:11:39.914 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-03-24T03:11:39.914 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-03-24T03:11:39.914 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 2, 2024-03-24T03:11:39.914 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-03-24T03:11:39.914 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-03-24T03:11:39.914 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 2, 2024-03-24T03:11:39.914 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 2, 2024-03-24T03:11:39.915 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-03-24T03:11:39.915 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-03-24T03:11:39.915 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 4194304, 2024-03-24T03:11:39.915 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 4196264, 2024-03-24T03:11:39.915 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-03-24T03:11:39.915 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-03-24T03:11:39.915 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 6, 2024-03-24T03:11:39.915 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.015838734, 2024-03-24T03:11:39.915 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002639789 2024-03-24T03:11:39.915 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.915 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-03-24T03:11:39.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.916 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-03-24T03:11:39.916 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 17, 2024-03-24T03:11:39.916 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 17, 2024-03-24T03:11:39.916 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 0, 2024-03-24T03:11:39.916 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 16, 2024-03-24T03:11:39.916 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 16, 2024-03-24T03:11:39.916 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 0, 2024-03-24T03:11:39.916 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 14, 2024-03-24T03:11:39.916 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 14, 2024-03-24T03:11:39.916 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 0, 2024-03-24T03:11:39.917 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 1, 2024-03-24T03:11:39.917 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 3, 2024-03-24T03:11:39.917 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 2, 2024-03-24T03:11:39.917 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 51620, 2024-03-24T03:11:39.917 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 223612 2024-03-24T03:11:39.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.917 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-03-24T03:11:39.917 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 4, 2024-03-24T03:11:39.917 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-03-24T03:11:39.917 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 4, 2024-03-24T03:11:39.918 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 17, 2024-03-24T03:11:39.918 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-03-24T03:11:39.918 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-03-24T03:11:39.918 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.918 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.918 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.918 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-03-24T03:11:39.918 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1, 2024-03-24T03:11:39.918 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000224978, 2024-03-24T03:11:39.918 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000224978 2024-03-24T03:11:39.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.919 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-03-24T03:11:39.919 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.919 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.919 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.919 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-03-24T03:11:39.919 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.919 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.919 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.920 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-03-24T03:11:39.920 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1, 2024-03-24T03:11:39.920 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000160800, 2024-03-24T03:11:39.920 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000160800 2024-03-24T03:11:39.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.920 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-03-24T03:11:39.920 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.920 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.920 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.921 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-03-24T03:11:39.921 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1, 2024-03-24T03:11:39.921 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000106691, 2024-03-24T03:11:39.921 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000106691 2024-03-24T03:11:39.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.921 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-03-24T03:11:39.921 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.921 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.921 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.922 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-03-24T03:11:39.922 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.922 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.922 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.922 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-03-24T03:11:39.922 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.922 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.922 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.922 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-03-24T03:11:39.923 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.923 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.923 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.923 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-03-24T03:11:39.923 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.923 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.923 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.924 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-03-24T03:11:39.924 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.924 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.924 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.925 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-03-24T03:11:39.925 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.925 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.925 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.925 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-03-24T03:11:39.925 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.925 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.925 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.925 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-03-24T03:11:39.926 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.926 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.926 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.926 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-03-24T03:11:39.926 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.926 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.926 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.926 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-03-24T03:11:39.927 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.927 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.927 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.927 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-03-24T03:11:39.927 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.927 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.927 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.927 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-03-24T03:11:39.927 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.928 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.928 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.928 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-03-24T03:11:39.928 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.928 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.928 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.928 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-03-24T03:11:39.928 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.928 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.928 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.929 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-03-24T03:11:39.929 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.929 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.929 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.929 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-03-24T03:11:39.929 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.929 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.929 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.930 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-03-24T03:11:39.930 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.930 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.930 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.930 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-03-24T03:11:39.930 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.930 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.930 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.930 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-03-24T03:11:39.931 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.931 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.931 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.931 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-03-24T03:11:39.931 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.931 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.931 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.931 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-03-24T03:11:39.932 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.932 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.932 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.932 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-03-24T03:11:39.932 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-03-24T03:11:39.932 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-03-24T03:11:39.932 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.932 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.932 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.932 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:39.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.933 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-03-24T03:11:39.933 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-03-24T03:11:39.933 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 3, 2024-03-24T03:11:39.933 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 2, 2024-03-24T03:11:39.933 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-03-24T03:11:39.933 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-03-24T03:11:39.933 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 2, 2024-03-24T03:11:39.933 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 1, 2024-03-24T03:11:39.933 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 632, 2024-03-24T03:11:39.933 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-03-24T03:11:39.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.934 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-03-24T03:11:39.934 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-03-24T03:11:39.934 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-03-24T03:11:39.934 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-03-24T03:11:39.934 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-03-24T03:11:39.934 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-03-24T03:11:39.934 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-03-24T03:11:39.934 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-24T03:11:39.934 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-03-24T03:11:39.934 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-24T03:11:39.935 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-03-24T03:11:39.935 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-03-24T03:11:39.935 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-03-24T03:11:39.935 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-24T03:11:39.935 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-03-24T03:11:39.935 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-03-24T03:11:39.935 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-03-24T03:11:39.935 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-03-24T03:11:39.935 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-03-24T03:11:39.935 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-24T03:11:39.935 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-03-24T03:11:39.935 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-24T03:11:39.936 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-03-24T03:11:39.936 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-03-24T03:11:39.936 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-03-24T03:11:39.936 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-03-24T03:11:39.936 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-03-24T03:11:39.936 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-24T03:11:39.936 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-03-24T03:11:39.936 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-03-24T03:11:39.936 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-03-24T03:11:39.936 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-03-24T03:11:39.936 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-03-24T03:11:39.937 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-03-24T03:11:39.937 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-03-24T03:11:39.937 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-03-24T03:11:39.937 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-03-24T03:11:39.937 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 238821, 2024-03-24T03:11:39.937 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 76, 2024-03-24T03:11:39.937 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-03-24T03:11:39.937 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-03-24T03:11:39.937 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-03-24T03:11:39.937 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-03-24T03:11:39.937 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-03-24T03:11:39.937 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-03-24T03:11:39.938 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-03-24T03:11:39.938 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-03-24T03:11:39.938 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 10240, 2024-03-24T03:11:39.938 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 216, 2024-03-24T03:11:39.938 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-03-24T03:11:39.938 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-03-24T03:11:39.938 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-03-24T03:11:39.938 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-03-24T03:11:39.938 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 68352, 2024-03-24T03:11:39.938 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 259, 2024-03-24T03:11:39.938 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-03-24T03:11:39.938 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-03-24T03:11:39.939 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-03-24T03:11:39.939 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-03-24T03:11:39.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.939 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-03-24T03:11:39.939 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-03-24T03:11:39.939 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-03-24T03:11:39.939 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 45, 2024-03-24T03:11:39.939 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 12497, 2024-03-24T03:11:39.939 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 33, 2024-03-24T03:11:39.939 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 45, 2024-03-24T03:11:39.939 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-03-24T03:11:39.939 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 45, 2024-03-24T03:11:39.940 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.229996127, 2024-03-24T03:11:39.940 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.005111025 2024-03-24T03:11:39.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.940 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-03-24T03:11:39.940 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-03-24T03:11:39.940 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 45, 2024-03-24T03:11:39.940 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 69 2024-03-24T03:11:39.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.940 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 45, 2024-03-24T03:11:39.940 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 0, 2024-03-24T03:11:39.940 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 45, 2024-03-24T03:11:39.940 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-03-24T03:11:39.941 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-03-24T03:11:39.941 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 0, 2024-03-24T03:11:39.941 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 10, 2024-03-24T03:11:39.941 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 0, 2024-03-24T03:11:39.941 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 6, 2024-03-24T03:11:39.941 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 8, 2024-03-24T03:11:39.941 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-03-24T03:11:39.941 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-03-24T03:11:39.941 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-03-24T03:11:39.941 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-03-24T03:11:39.941 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 6, 2024-03-24T03:11:39.942 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-03-24T03:11:39.942 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-03-24T03:11:39.942 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-03-24T03:11:39.942 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 0, 2024-03-24T03:11:39.942 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 20, 2024-03-24T03:11:39.942 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-03-24T03:11:39.942 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-03-24T03:11:39.942 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-03-24T03:11:39.942 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-03-24T03:11:39.942 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-03-24T03:11:39.942 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-03-24T03:11:39.943 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-03-24T03:11:39.943 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-03-24T03:11:39.943 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-03-24T03:11:39.943 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-03-24T03:11:39.943 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-03-24T03:11:39.943 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-03-24T03:11:39.943 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-03-24T03:11:39.943 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-03-24T03:11:39.943 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-03-24T03:11:39.943 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-03-24T03:11:39.943 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-03-24T03:11:39.943 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-03-24T03:11:39.944 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-03-24T03:11:39.944 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-03-24T03:11:39.944 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-03-24T03:11:39.944 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-03-24T03:11:39.944 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-03-24T03:11:39.944 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-03-24T03:11:39.944 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-03-24T03:11:39.944 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-03-24T03:11:39.944 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 103, 2024-03-24T03:11:39.944 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-03-24T03:11:39.944 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 14, 2024-03-24T03:11:39.944 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 8, 2024-03-24T03:11:39.945 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 8, 2024-03-24T03:11:39.945 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-03-24T03:11:39.945 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-03-24T03:11:39.945 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 17, 2024-03-24T03:11:39.945 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 0, 2024-03-24T03:11:39.945 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 2 2024-03-24T03:11:39.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.945 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-03-24T03:11:39.945 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-03-24T03:11:39.945 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-03-24T03:11:39.945 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-03-24T03:11:39.945 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-03-24T03:11:39.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.946 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-03-24T03:11:39.946 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-03-24T03:11:39.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.946 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-03-24T03:11:39.946 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-03-24T03:11:39.946 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-03-24T03:11:39.946 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-03-24T03:11:39.946 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-03-24T03:11:39.946 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-03-24T03:11:39.946 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-03-24T03:11:39.947 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-03-24T03:11:39.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.947 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-24T03:11:39.947 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:39.947 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-03-24T03:11:39.947 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:39.947 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 633, 2024-03-24T03:11:39.947 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 1520627, 2024-03-24T03:11:39.947 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:39.947 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 633, 2024-03-24T03:11:39.947 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-03-24T03:11:39.947 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-03-24T03:11:39.948 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 633, 2024-03-24T03:11:39.948 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1520627, 2024-03-24T03:11:39.948 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:39.948 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.948 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.948 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.948 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:39.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.948 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-03-24T03:11:39.948 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:39.948 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-03-24T03:11:39.948 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:39.949 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-03-24T03:11:39.949 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-03-24T03:11:39.949 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:39.949 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:39.949 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 45, 2024-03-24T03:11:39.949 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 12497, 2024-03-24T03:11:39.949 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 45, 2024-03-24T03:11:39.949 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 12497, 2024-03-24T03:11:39.949 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:39.949 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.949 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.949 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.950 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:39.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.950 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-03-24T03:11:39.950 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:39.950 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-03-24T03:11:39.950 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:39.950 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-03-24T03:11:39.950 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-03-24T03:11:39.950 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:39.950 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:39.950 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 45, 2024-03-24T03:11:39.951 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 45, 2024-03-24T03:11:39.951 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 45, 2024-03-24T03:11:39.951 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 45, 2024-03-24T03:11:39.951 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:39.951 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.951 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.951 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.951 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:39.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.951 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-03-24T03:11:39.951 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:39.951 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-03-24T03:11:39.952 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:39.952 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-03-24T03:11:39.952 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-03-24T03:11:39.952 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:39.952 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:39.952 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-03-24T03:11:39.952 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-03-24T03:11:39.952 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-03-24T03:11:39.952 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-03-24T03:11:39.952 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:39.952 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.953 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.953 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.953 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:39.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:39.953 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x564d8ff4e500": { 2024-03-24T03:11:39.953 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:39.953 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-03-24T03:11:39.953 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:39.953 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 7, 2024-03-24T03:11:39.953 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 1960, 2024-03-24T03:11:39.953 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:39.953 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 7, 2024-03-24T03:11:39.954 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-03-24T03:11:39.954 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-03-24T03:11:39.954 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 6, 2024-03-24T03:11:39.954 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1960, 2024-03-24T03:11:39.954 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:39.954 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:39.954 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:39.954 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:39.954 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:39.954 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:39.954 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-03-24T03:11:39.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:39 smithi017 ceph-mon[31926]: pgmap v329: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:40.734 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-24T03:11:40.734 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-24T03:11:40.734 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-24T03:11:40.735 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=6 2024-03-24T03:11:40.735 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-24T03:11:40.735 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-24T03:11:40.735 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-24T03:11:40.735 DEBUG:tasks.check_counter:Getting stats from i 2024-03-24T03:11:40.735 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-24T03:11:40.991 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T03:11:41.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:41 smithi188 ceph-mon[36206]: pgmap v330: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:41.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:41 smithi019 ceph-mon[35944]: pgmap v330: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:41.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:41 smithi017 ceph-mon[31926]: pgmap v330: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:42.793 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-03-24T03:11:42.794 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-03-24T03:11:42.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 5957, 2024-03-24T03:11:42.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 5497, 2024-03-24T03:11:42.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 1743230, 2024-03-24T03:11:42.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 10149016, 2024-03-24T03:11:42.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 11, 2024-03-24T03:11:42.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-03-24T03:11:42.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.643545728, 2024-03-24T03:11:42.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.233115532, 2024-03-24T03:11:42.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.121567221, 2024-03-24T03:11:42.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.201864118, 2024-03-24T03:11:42.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 5498, 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 2.152790972, 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000391558 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1542, 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000247876, 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000160 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 42528, 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 1136 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-03-24T03:11:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 3476, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 3375, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 979770, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 4792561, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 13, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 7, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.338880884, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.111386268, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.071743102, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.110303028, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3372, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1.669198767, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000495017 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:42.796 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 825, 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000134496, 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000163 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 42544, 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 455440 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 8324, 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 8657, 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 4070161, 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 3039273, 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 19, 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 8, 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.589204893, 2024-03-24T03:11:42.797 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.204144843, 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.478835806, 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.101322509, 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 8645, 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 2.398567379, 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000277451 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 5478, 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.002943549, 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000537 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 1877408, 2024-03-24T03:11:42.798 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 266880 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3652, 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 2.950674125, 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000807961 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-03-24T03:11:42.799 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3922, 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.206423556, 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000052632 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 193, 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 193, 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 193, 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.186067273, 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000964079 2024-03-24T03:11:42.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 0, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 12, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 35, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 2, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 33, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 32, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 2, 2024-03-24T03:11:42.801 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 2, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 2, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 280, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 224, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 0, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-03-24T03:11:42.802 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 4, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 234, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 6, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-03-24T03:11:42.803 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 22, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 1767, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 4313, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 4276, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 4, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 27 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 9, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-03-24T03:11:42.804 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 0, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 21, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 2, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 5, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 3, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-03-24T03:11:42.805 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 4411, 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 4411, 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-03-24T03:11:42.806 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 7, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 7, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 2, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 4194304, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 15886966, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 4377, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 755.293640781, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.172559662 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-03-24T03:11:42.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 26, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 29, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 3, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 20, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 20, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 0, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 35, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 38, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 3, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 2, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 22, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 20, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 60072, 2024-03-24T03:11:42.808 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 231804 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 254, 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 193, 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 40, 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 7, 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.012713676, 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001816239 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 110, 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.020133039, 2024-03-24T03:11:42.809 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000183027 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 24, 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.006319010, 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000263292 2024-03-24T03:11:42.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.811 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 10, 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.001548275, 2024-03-24T03:11:42.812 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000154827 2024-03-24T03:11:42.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.813 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-03-24T03:11:42.813 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 8, 2024-03-24T03:11:42.813 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.019282158, 2024-03-24T03:11:42.813 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002410269 2024-03-24T03:11:42.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.813 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-03-24T03:11:42.813 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 10, 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.053478844, 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.005347884 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-03-24T03:11:42.814 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2, 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000823478, 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000411739 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2, 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.006749255, 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003374627 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.815 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2, 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.010779429, 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.005389714 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 4, 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.019661110, 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.004915277 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.816 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2, 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.001042562, 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000521281 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 9, 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.023780816, 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002642312 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3, 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.009755621, 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003251873 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 2, 2024-03-24T03:11:42.819 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 7, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 5, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 2, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 27, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 13, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 614, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-03-24T03:11:42.820 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-03-24T03:11:42.821 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-03-24T03:11:42.822 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 379758, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 158, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 792, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 9, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 10240, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 216, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-03-24T03:11:42.823 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 125362, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 1009, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 9590, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 11710354, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 32, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 9590, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 9590, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 797.246477169, 2024-03-24T03:11:42.824 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.083133105 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 9590, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 9636 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 9590, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 3, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 9587, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 0, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 11, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 3, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 4380, 2024-03-24T03:11:42.825 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 30, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 5127, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 0, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 22, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-03-24T03:11:42.826 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-03-24T03:11:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 103, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 19, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 12, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 12, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 57, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 0, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 6 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.828 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 3585, 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 1, 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 4613, 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 3, 2024-03-24T03:11:42.829 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 17753, 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 5209534, 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 17753, 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 17753, 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 5209534, 2024-03-24T03:11:42.830 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:42.831 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.831 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.831 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.831 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:42.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.831 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-03-24T03:11:42.831 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:42.831 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-03-24T03:11:42.831 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:42.831 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-03-24T03:11:42.831 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-03-24T03:11:42.831 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 9590, 2024-03-24T03:11:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 11710689, 2024-03-24T03:11:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 9590, 2024-03-24T03:11:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 11710689, 2024-03-24T03:11:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 9590, 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 9590, 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 9590, 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 9590, 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:42.834 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 3, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 335, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 3, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 3, 2024-03-24T03:11:42.835 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 335, 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x562131212500": { 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 4411, 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 11692662, 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 4411, 2024-03-24T03:11:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-03-24T03:11:42.837 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-03-24T03:11:42.837 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 4377, 2024-03-24T03:11:42.837 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 11692662, 2024-03-24T03:11:42.837 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:42.837 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:42.837 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:42.837 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:42.837 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:42.837 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:42.837 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-03-24T03:11:43.593 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-24T03:11:43.594 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-24T03:11:43.594 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=9 2024-03-24T03:11:43.594 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-03-24T03:11:43.594 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-03-24T03:11:43.594 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=10 2024-03-24T03:11:43.594 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=4 2024-03-24T03:11:43.594 DEBUG:tasks.check_counter:Getting stats from l 2024-03-24T03:11:43.594 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2a682800-e989-11ee-95cd-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-24T03:11:43.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:43 smithi188 ceph-mon[36206]: pgmap v331: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:43.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:43 smithi019 ceph-mon[35944]: pgmap v331: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:43.911 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/mon.c/config 2024-03-24T03:11:43.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:43 smithi017 ceph-mon[31926]: pgmap v331: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:45.616 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:45 smithi188 ceph-mon[36206]: pgmap v332: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:45.616 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-03-24T03:11:45.616 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-03-24T03:11:45.616 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1016, 2024-03-24T03:11:45.616 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1005, 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 1730144, 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 379304, 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 5, 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 1, 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.092231846, 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.025622443, 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.048145356, 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.023642281, 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1004, 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.026192544, 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000026088 2024-03-24T03:11:45.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 1473440, 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 136352 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 4, 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 172, 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 876, 2024-03-24T03:11:45.618 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 448938, 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 6, 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.008170810, 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.005030409, 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.001660952, 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000061221, 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 169, 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.019621865, 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000116105 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:45.619 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 448240 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1410, 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1410, 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 363104, 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 447563, 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-03-24T03:11:45.620 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 3, 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.077177350, 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.022720063, 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.019715737, 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.030280641, 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1409, 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.031159568, 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000022114 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 448 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3691, 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.077214336, 2024-03-24T03:11:45.622 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000020919 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 0, 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 0, 2024-03-24T03:11:45.623 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 0, 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 0, 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 0, 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 0, 2024-03-24T03:11:45.624 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 1, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-03-24T03:11:45.625 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-03-24T03:11:45.626 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-03-24T03:11:45.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 0, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-03-24T03:11:45.628 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 0, 2024-03-24T03:11:45.629 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 4, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 0, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 2, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 2, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 4194304, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 4195591, 2024-03-24T03:11:45.630 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 4195200, 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 4 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 3, 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 3, 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 0, 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 1, 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 1, 2024-03-24T03:11:45.631 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 0, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 0, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 0, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 0, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 0, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 0, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 46432, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 215420 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 0, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 0, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 0, 2024-03-24T03:11:45.632 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.634 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.635 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.636 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.637 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-03-24T03:11:45.637 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.637 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.637 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.637 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.639 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.641 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.642 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.643 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 2, 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 2, 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 0, 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 2, 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 0, 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 0, 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 2064, 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-03-24T03:11:45.644 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-03-24T03:11:45.645 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-03-24T03:11:45.646 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 182059, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 48, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 10240, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 216, 2024-03-24T03:11:45.647 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 8320, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 13, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 2172, 2024-03-24T03:11:45.648 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 21, 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 3, 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 2172, 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2172, 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 4.330926867, 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001993981 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2172, 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 2173 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 2172, 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 2172, 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 0, 2024-03-24T03:11:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 1400, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 770, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-03-24T03:11:45.650 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 1, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-03-24T03:11:45.651 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 103, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-03-24T03:11:45.652 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 14, 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 4, 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 4, 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 0, 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 2, 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 0 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 0, 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.653 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:45.654 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 2426, 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 1859800, 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 2426, 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 2426, 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1859800, 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-03-24T03:11:45.655 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 2172, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 1292, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 2172, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1292, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:45.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 2172, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 2172, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 2172, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 2172, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.657 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-03-24T03:11:45.658 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x55c0fda6e5a0": { 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-03-24T03:11:45.659 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-03-24T03:11:45.660 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-03-24T03:11:45.660 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-03-24T03:11:45.660 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-03-24T03:11:45.660 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-03-24T03:11:45.660 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-03-24T03:11:45.660 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:45.660 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-03-24T03:11:45.660 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-03-24T03:11:45.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:45 smithi019 ceph-mon[35944]: pgmap v332: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:45.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:45 smithi017 ceph-mon[31926]: pgmap v332: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:46.427 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-24T03:11:46.427 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-24T03:11:46.427 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-03-24T03:11:46.427 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-03-24T03:11:46.427 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-03-24T03:11:46.427 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-24T03:11:46.427 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-24T03:11:46.428 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-24T03:11:46.438 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-24T03:11:46.439 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T03:11:46.439 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T03:11:46.460 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-24T03:11:46.460 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T03:11:46.519 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T03:11:46.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.679+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/3796541824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ff4101010 msgr2=0x7f0ff41054d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:46.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.679+0000 7f0ffbe8a640 1 --2- 172.21.15.17:0/3796541824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ff4101010 0x7f0ff41054d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009930 tx=0x7f0fe402bcd0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.679+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/3796541824 shutdown_connections 2024-03-24T03:11:46.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.679+0000 7f0ffbe8a640 1 --2- 172.21.15.17:0/3796541824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ff4105af0 0x7f0ff4105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.679+0000 7f0ffbe8a640 1 --2- 172.21.15.17:0/3796541824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ff4101010 0x7f0ff41054d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.679+0000 7f0ffbe8a640 1 --2- 172.21.15.17:0/3796541824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff41006f0 0x7f0ff4100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.679+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/3796541824 >> 172.21.15.17:0/3796541824 conn(0x7f0ff4076920 msgr2=0x7f0ff4076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:46.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.679+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/3796541824 shutdown_connections 2024-03-24T03:11:46.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.679+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/3796541824 wait complete. 2024-03-24T03:11:46.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ffbe8a640 1 Processor -- start 2024-03-24T03:11:46.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ffbe8a640 1 -- start start 2024-03-24T03:11:46.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ffbe8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ff41006f0 0x7f0ff419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:46.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ffbe8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ff4101010 0x7f0ff419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:46.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ffbe8a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff4105af0 0x7f0ff41a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:46.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ffbe8a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4118080 con 0x7f0ff4101010 2024-03-24T03:11:46.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ffbe8a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4117f00 con 0x7f0ff41006f0 2024-03-24T03:11:46.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ffbe8a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4118200 con 0x7f0ff4105af0 2024-03-24T03:11:46.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ff9425640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ff4101010 0x7f0ff419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:46.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ff9425640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ff4101010 0x7f0ff419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44094/0 (socket says 172.21.15.17:44094) 2024-03-24T03:11:46.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ff9425640 1 -- 172.21.15.17:0/2374250535 learned_addr learned my addr 172.21.15.17:0/2374250535 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:46.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ff9c26640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ff41006f0 0x7f0ff419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:46.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ffa427640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff4105af0 0x7f0ff41a2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:46.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ff9425640 1 -- 172.21.15.17:0/2374250535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff4105af0 msgr2=0x7f0ff41a2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:46.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ff9425640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff4105af0 0x7f0ff41a2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ff9425640 1 -- 172.21.15.17:0/2374250535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ff41006f0 msgr2=0x7f0ff419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:46.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ff9425640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ff41006f0 0x7f0ff419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ff9425640 1 -- 172.21.15.17:0/2374250535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff41a3610 con 0x7f0ff4101010 2024-03-24T03:11:46.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ff9c26640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ff41006f0 0x7f0ff419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:11:46.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.680+0000 7f0ff9425640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ff4101010 0x7f0ff419eb60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009230 tx=0x7f0fe402cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:46.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.681+0000 7f0feaffd640 1 -- 172.21.15.17:0/2374250535 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4041070 con 0x7f0ff4101010 2024-03-24T03:11:46.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.681+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/2374250535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff41a38a0 con 0x7f0ff4101010 2024-03-24T03:11:46.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.681+0000 7f0feaffd640 1 -- 172.21.15.17:0/2374250535 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fe4004450 con 0x7f0ff4101010 2024-03-24T03:11:46.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.681+0000 7f0feaffd640 1 -- 172.21.15.17:0/2374250535 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4003cd0 con 0x7f0ff4101010 2024-03-24T03:11:46.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.681+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/2374250535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff41ab190 con 0x7f0ff4101010 2024-03-24T03:11:46.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.682+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/2374250535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff406a4f0 con 0x7f0ff4101010 2024-03-24T03:11:46.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.683+0000 7f0feaffd640 1 -- 172.21.15.17:0/2374250535 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0fe4044070 con 0x7f0ff4101010 2024-03-24T03:11:46.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.683+0000 7f0feaffd640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f0fe00a2970 0x7f0fe00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:46.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.684+0000 7f0feaffd640 1 -- 172.21.15.17:0/2374250535 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9708+0+0 (secure 0 0 0) 0x7f0fe40f1000 con 0x7f0ff4101010 2024-03-24T03:11:46.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.684+0000 7f0ff9c26640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f0fe00a2970 0x7f0fe00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:46.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.684+0000 7f0ff9c26640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f0fe00a2970 0x7f0fe00a4e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009fc0 tx=0x7f0ff0005f90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:46.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.687+0000 7f0feaffd640 1 -- 172.21.15.17:0/2374250535 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0fe40bd9d0 con 0x7f0ff4101010 2024-03-24T03:11:46.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.872+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/2374250535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ff41ab360 con 0x7f0ff4101010 2024-03-24T03:11:46.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.873+0000 7f0feaffd640 1 -- 172.21.15.17:0/2374250535 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+25580 (secure 0 0 0) 0x7f0fe40bd2d0 con 0x7f0ff4101010 2024-03-24T03:11:46.873 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:11:46.874 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":103,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T03:11:15.563541+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/934316425":"2024-03-25T03:01:04.109518+0000","172.21.15.17:6800/2663792883":"2024-03-25T03:01:04.109518+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/1698088665":"2024-03-25T03:01:04.109518+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/2663792883":"2024-03-25T03:01:04.109518+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/3359268740":"2024-03-25T03:01:04.109518+0000","172.21.15.17:0/1112257866":"2024-03-24T04:05:56.644241+0000","172.21.15.17:0/1127409383":"2024-03-25T03:01:04.109518+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T03:11:46.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.875+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/2374250535 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f0fe00a2970 msgr2=0x7f0fe00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:46.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.875+0000 7f0ffbe8a640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f0fe00a2970 0x7f0fe00a4e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009fc0 tx=0x7f0ff0005f90 comp rx=0 tx=0).stop 2024-03-24T03:11:46.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.875+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/2374250535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ff4101010 msgr2=0x7f0ff419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:46.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.875+0000 7f0ffbe8a640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ff4101010 0x7f0ff419eb60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009230 tx=0x7f0fe402cbc0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.875+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/2374250535 shutdown_connections 2024-03-24T03:11:46.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.875+0000 7f0ffbe8a640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f0fe00a2970 0x7f0fe00a4e30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.875+0000 7f0ffbe8a640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff4105af0 0x7f0ff41a2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.875+0000 7f0ffbe8a640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ff4101010 0x7f0ff419eb60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.875+0000 7f0ffbe8a640 1 --2- 172.21.15.17:0/2374250535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ff41006f0 0x7f0ff419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.875+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/2374250535 >> 172.21.15.17:0/2374250535 conn(0x7f0ff4076920 msgr2=0x7f0ff41034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:46.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.876+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/2374250535 shutdown_connections 2024-03-24T03:11:46.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.876+0000 7f0ffbe8a640 1 -- 172.21.15.17:0/2374250535 wait complete. 2024-03-24T03:11:46.886 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T02:53:54.207139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-24T02:58:34.785192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-24T02:58:35.679772+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-24T02:58:39.221772+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-24T03:11:46.887 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T03:11:46.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.994+0000 7f7a7d60b640 1 -- 172.21.15.17:0/4250163059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a78101630 msgr2=0x7f7a78101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.994+0000 7f7a7d60b640 1 --2- 172.21.15.17:0/4250163059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a78101630 0x7f7a78101ab0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009930 tx=0x7f7a6002bc30 comp rx=0 tx=0).stop 2024-03-24T03:11:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.994+0000 7f7a7d60b640 1 -- 172.21.15.17:0/4250163059 shutdown_connections 2024-03-24T03:11:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.994+0000 7f7a7d60b640 1 --2- 172.21.15.17:0/4250163059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a78101ff0 0x7f7a7810e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.994+0000 7f7a7d60b640 1 --2- 172.21.15.17:0/4250163059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a78101630 0x7f7a78101ab0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.994+0000 7f7a7d60b640 1 --2- 172.21.15.17:0/4250163059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a78107640 0x7f7a78107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.994+0000 7f7a7d60b640 1 -- 172.21.15.17:0/4250163059 >> 172.21.15.17:0/4250163059 conn(0x7f7a78078aa0 msgr2=0x7f7a78078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.994+0000 7f7a7d60b640 1 -- 172.21.15.17:0/4250163059 shutdown_connections 2024-03-24T03:11:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.994+0000 7f7a7d60b640 1 -- 172.21.15.17:0/4250163059 wait complete. 2024-03-24T03:11:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.994+0000 7f7a7d60b640 1 Processor -- start 2024-03-24T03:11:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.994+0000 7f7a7d60b640 1 -- start start 2024-03-24T03:11:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a7d60b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a78101630 0x7f7a78197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a7d60b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a78101ff0 0x7f7a781984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a7d60b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a78107640 0x7f7a7819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a7d60b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78111aa0 con 0x7f7a78107640 2024-03-24T03:11:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a7d60b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78111920 con 0x7f7a78101ff0 2024-03-24T03:11:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a7d60b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78111c20 con 0x7f7a78101630 2024-03-24T03:11:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a78107640 0x7f7a7819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a76ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a78101630 0x7f7a78197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a78107640 0x7f7a7819c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44110/0 (socket says 172.21.15.17:44110) 2024-03-24T03:11:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a777fe640 1 -- 172.21.15.17:0/2765713616 learned_addr learned my addr 172.21.15.17:0/2765713616 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a767fc640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a78101ff0 0x7f7a781984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a777fe640 1 -- 172.21.15.17:0/2765713616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a78101630 msgr2=0x7f7a78197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a777fe640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a78101630 0x7f7a78197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a777fe640 1 -- 172.21.15.17:0/2765713616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a78101ff0 msgr2=0x7f7a781984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a777fe640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a78101ff0 0x7f7a781984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a777fe640 1 -- 172.21.15.17:0/2765713616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7819cf70 con 0x7f7a78107640 2024-03-24T03:11:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a767fc640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a78101ff0 0x7f7a781984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:11:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a777fe640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a78107640 0x7f7a7819c840 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800efd0 tx=0x7f7a6800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a6ffff640 1 -- 172.21.15.17:0/2765713616 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68003b70 con 0x7f7a78107640 2024-03-24T03:11:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a6ffff640 1 -- 172.21.15.17:0/2765713616 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a68003d10 con 0x7f7a78107640 2024-03-24T03:11:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a7d60b640 1 -- 172.21.15.17:0/2765713616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7819d260 con 0x7f7a78107640 2024-03-24T03:11:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a6ffff640 1 -- 172.21.15.17:0/2765713616 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a680109f0 con 0x7f7a78107640 2024-03-24T03:11:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.995+0000 7f7a7d60b640 1 -- 172.21.15.17:0/2765713616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a781a4af0 con 0x7f7a78107640 2024-03-24T03:11:46.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.996+0000 7f7a7d60b640 1 -- 172.21.15.17:0/2765713616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a780688e0 con 0x7f7a78107640 2024-03-24T03:11:46.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.997+0000 7f7a6ffff640 1 -- 172.21.15.17:0/2765713616 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7a68004020 con 0x7f7a78107640 2024-03-24T03:11:46.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.998+0000 7f7a6ffff640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f7a640a2970 0x7f7a640a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.998+0000 7f7a76ffd640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f7a640a2970 0x7f7a640a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.998+0000 7f7a6ffff640 1 -- 172.21.15.17:0/2765713616 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9708+0+0 (secure 0 0 0) 0x7f7a680cf710 con 0x7f7a78107640 2024-03-24T03:11:47.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:46.998+0000 7f7a76ffd640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f7a640a2970 0x7f7a640a4e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009b50 tx=0x7f7a70005f90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:47.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.002+0000 7f7a6ffff640 1 -- 172.21.15.17:0/2765713616 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7a6809bfd0 con 0x7f7a78107640 2024-03-24T03:11:47.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.188+0000 7f7a7d60b640 1 -- 172.21.15.17:0/2765713616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a7802d070 con 0x7f7a78107640 2024-03-24T03:11:47.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.188+0000 7f7a6ffff640 1 -- 172.21.15.17:0/2765713616 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a6809b8d0 con 0x7f7a78107640 2024-03-24T03:11:47.188 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-24T03:11:47.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.190+0000 7f7a7d60b640 1 -- 172.21.15.17:0/2765713616 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f7a640a2970 msgr2=0x7f7a640a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.190+0000 7f7a7d60b640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f7a640a2970 0x7f7a640a4e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009b50 tx=0x7f7a70005f90 comp rx=0 tx=0).stop 2024-03-24T03:11:47.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.190+0000 7f7a7d60b640 1 -- 172.21.15.17:0/2765713616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a78107640 msgr2=0x7f7a7819c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.190+0000 7f7a7d60b640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a78107640 0x7f7a7819c840 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800efd0 tx=0x7f7a6800c6a0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.190+0000 7f7a7d60b640 1 -- 172.21.15.17:0/2765713616 shutdown_connections 2024-03-24T03:11:47.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.190+0000 7f7a7d60b640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f7a640a2970 0x7f7a640a4e30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.190+0000 7f7a7d60b640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a78107640 0x7f7a7819c840 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.190+0000 7f7a7d60b640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a78101ff0 0x7f7a781984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.190+0000 7f7a7d60b640 1 --2- 172.21.15.17:0/2765713616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a78101630 0x7f7a78197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.190+0000 7f7a7d60b640 1 -- 172.21.15.17:0/2765713616 >> 172.21.15.17:0/2765713616 conn(0x7f7a78078aa0 msgr2=0x7f7a78104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:47.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.190+0000 7f7a7d60b640 1 -- 172.21.15.17:0/2765713616 shutdown_connections 2024-03-24T03:11:47.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.190+0000 7f7a7d60b640 1 -- 172.21.15.17:0/2765713616 wait complete. 2024-03-24T03:11:47.200 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-24T03:11:47.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.307+0000 7f0dfc692640 1 -- 172.21.15.17:0/2533281325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0df4107640 msgr2=0x7f0df4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.307+0000 7f0dfc692640 1 --2- 172.21.15.17:0/2533281325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0df4107640 0x7f0df4107a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009900 tx=0x7f0df002bc30 comp rx=0 tx=0).stop 2024-03-24T03:11:47.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.307+0000 7f0dfc692640 1 -- 172.21.15.17:0/2533281325 shutdown_connections 2024-03-24T03:11:47.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.307+0000 7f0dfc692640 1 --2- 172.21.15.17:0/2533281325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0df4101ff0 0x7f0df410e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.307+0000 7f0dfc692640 1 --2- 172.21.15.17:0/2533281325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0df4101630 0x7f0df4101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.307+0000 7f0dfc692640 1 --2- 172.21.15.17:0/2533281325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0df4107640 0x7f0df4107a20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.307+0000 7f0dfc692640 1 -- 172.21.15.17:0/2533281325 >> 172.21.15.17:0/2533281325 conn(0x7f0df4078aa0 msgr2=0x7f0df4078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.307+0000 7f0dfc692640 1 -- 172.21.15.17:0/2533281325 shutdown_connections 2024-03-24T03:11:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.307+0000 7f0dfc692640 1 -- 172.21.15.17:0/2533281325 wait complete. 2024-03-24T03:11:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfc692640 1 Processor -- start 2024-03-24T03:11:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfc692640 1 -- start start 2024-03-24T03:11:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfc692640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0df4101630 0x7f0df4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfc692640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0df4101ff0 0x7f0df4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfc692640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0df4107640 0x7f0df419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfc692640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df4111a10 con 0x7f0df4101630 2024-03-24T03:11:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfc692640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df4111890 con 0x7f0df4107640 2024-03-24T03:11:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfc692640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df4111b90 con 0x7f0df4101ff0 2024-03-24T03:11:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfa42e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0df4101630 0x7f0df4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfa42e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0df4101630 0x7f0df4197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44126/0 (socket says 172.21.15.17:44126) 2024-03-24T03:11:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfa42e640 1 -- 172.21.15.17:0/2595902226 learned_addr learned my addr 172.21.15.17:0/2595902226 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfac2f640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0df4107640 0x7f0df419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0df9c2d640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0df4101ff0 0x7f0df4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfa42e640 1 -- 172.21.15.17:0/2595902226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0df4101ff0 msgr2=0x7f0df4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfa42e640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0df4101ff0 0x7f0df4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfa42e640 1 -- 172.21.15.17:0/2595902226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0df4107640 msgr2=0x7f0df419c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfa42e640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0df4107640 0x7f0df419c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.308+0000 7f0dfa42e640 1 -- 172.21.15.17:0/2595902226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df419cee0 con 0x7f0df4101630 2024-03-24T03:11:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.309+0000 7f0dfa42e640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0df4101630 0x7f0df4197ef0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0df0005940 tx=0x7f0df0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.309+0000 7f0deb7fe640 1 -- 172.21.15.17:0/2595902226 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0041070 con 0x7f0df4101630 2024-03-24T03:11:47.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.309+0000 7f0dfc692640 1 -- 172.21.15.17:0/2595902226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df419d170 con 0x7f0df4101630 2024-03-24T03:11:47.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.309+0000 7f0deb7fe640 1 -- 172.21.15.17:0/2595902226 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0df00054e0 con 0x7f0df4101630 2024-03-24T03:11:47.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.309+0000 7f0deb7fe640 1 -- 172.21.15.17:0/2595902226 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0003c30 con 0x7f0df4101630 2024-03-24T03:11:47.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.309+0000 7f0dfc692640 1 -- 172.21.15.17:0/2595902226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df41a4a60 con 0x7f0df4101630 2024-03-24T03:11:47.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.310+0000 7f0dfc692640 1 -- 172.21.15.17:0/2595902226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df40688e0 con 0x7f0df4101630 2024-03-24T03:11:47.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.311+0000 7f0deb7fe640 1 -- 172.21.15.17:0/2595902226 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0df0044070 con 0x7f0df4101630 2024-03-24T03:11:47.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.311+0000 7f0deb7fe640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f0de00a2970 0x7f0de00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.311+0000 7f0deb7fe640 1 -- 172.21.15.17:0/2595902226 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9708+0+0 (secure 0 0 0) 0x7f0df00f0bb0 con 0x7f0df4101630 2024-03-24T03:11:47.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.311+0000 7f0df9c2d640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f0de00a2970 0x7f0de00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.312+0000 7f0df9c2d640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f0de00a2970 0x7f0de00a4e30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0df4105cd0 tx=0x7f0de4005f90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:47.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.316+0000 7f0deb7fe640 1 -- 172.21.15.17:0/2595902226 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0df00bd580 con 0x7f0df4101630 2024-03-24T03:11:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.498+0000 7f0dfc692640 1 -- 172.21.15.17:0/2595902226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0df4109ba0 con 0x7f0df4101630 2024-03-24T03:11:47.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.498+0000 7f0deb7fe640 1 -- 172.21.15.17:0/2595902226 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0df00bce80 con 0x7f0df4101630 2024-03-24T03:11:47.498 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T03:11:47.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.500+0000 7f0dfc692640 1 -- 172.21.15.17:0/2595902226 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f0de00a2970 msgr2=0x7f0de00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.500+0000 7f0dfc692640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f0de00a2970 0x7f0de00a4e30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0df4105cd0 tx=0x7f0de4005f90 comp rx=0 tx=0).stop 2024-03-24T03:11:47.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.500+0000 7f0dfc692640 1 -- 172.21.15.17:0/2595902226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0df4101630 msgr2=0x7f0df4197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.500+0000 7f0dfc692640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0df4101630 0x7f0df4197ef0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0df0005940 tx=0x7f0df0004270 comp rx=0 tx=0).stop 2024-03-24T03:11:47.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.500+0000 7f0dfc692640 1 -- 172.21.15.17:0/2595902226 shutdown_connections 2024-03-24T03:11:47.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.500+0000 7f0dfc692640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f0de00a2970 0x7f0de00a4e30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.500+0000 7f0dfc692640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0df4107640 0x7f0df419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.500+0000 7f0dfc692640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0df4101ff0 0x7f0df4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.500+0000 7f0dfc692640 1 --2- 172.21.15.17:0/2595902226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0df4101630 0x7f0df4197ef0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.500+0000 7f0dfc692640 1 -- 172.21.15.17:0/2595902226 >> 172.21.15.17:0/2595902226 conn(0x7f0df4078aa0 msgr2=0x7f0df4103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:47.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.500+0000 7f0dfc692640 1 -- 172.21.15.17:0/2595902226 shutdown_connections 2024-03-24T03:11:47.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.500+0000 7f0dfc692640 1 -- 172.21.15.17:0/2595902226 wait complete. 2024-03-24T03:11:47.511 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-24T03:11:47.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.617+0000 7efe38d5b640 1 -- 172.21.15.17:0/371359942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe34107710 msgr2=0x7efe34107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.617+0000 7efe38d5b640 1 --2- 172.21.15.17:0/371359942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe34107710 0x7efe34107af0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009930 tx=0x7efe2c02bc30 comp rx=0 tx=0).stop 2024-03-24T03:11:47.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.617+0000 7efe38d5b640 1 -- 172.21.15.17:0/371359942 shutdown_connections 2024-03-24T03:11:47.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.617+0000 7efe38d5b640 1 --2- 172.21.15.17:0/371359942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3406a710 0x7efe3410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.617+0000 7efe38d5b640 1 --2- 172.21.15.17:0/371359942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe34069d50 0x7efe3406a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.617+0000 7efe38d5b640 1 --2- 172.21.15.17:0/371359942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe34107710 0x7efe34107af0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.617+0000 7efe38d5b640 1 -- 172.21.15.17:0/371359942 >> 172.21.15.17:0/371359942 conn(0x7efe34077670 msgr2=0x7efe34077a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:47.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.617+0000 7efe38d5b640 1 -- 172.21.15.17:0/371359942 shutdown_connections 2024-03-24T03:11:47.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.617+0000 7efe38d5b640 1 -- 172.21.15.17:0/371359942 wait complete. 2024-03-24T03:11:47.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.618+0000 7efe38d5b640 1 Processor -- start 2024-03-24T03:11:47.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.618+0000 7efe38d5b640 1 -- start start 2024-03-24T03:11:47.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.618+0000 7efe38d5b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe34069d50 0x7efe34197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.618+0000 7efe38d5b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe3406a710 0x7efe34198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.618+0000 7efe38d5b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe34107710 0x7efe3419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.618+0000 7efe38d5b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe34111ab0 con 0x7efe3406a710 2024-03-24T03:11:47.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.618+0000 7efe38d5b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe34111930 con 0x7efe34069d50 2024-03-24T03:11:47.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.618+0000 7efe38d5b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe34111c30 con 0x7efe34107710 2024-03-24T03:11:47.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.618+0000 7efe31d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe3406a710 0x7efe34198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.618+0000 7efe3259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe34069d50 0x7efe34197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.618+0000 7efe31d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe3406a710 0x7efe34198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44134/0 (socket says 172.21.15.17:44134) 2024-03-24T03:11:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe31d9b640 1 -- 172.21.15.17:0/3966858095 learned_addr learned my addr 172.21.15.17:0/3966858095 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe32d9d640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe34107710 0x7efe3419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe31d9b640 1 -- 172.21.15.17:0/3966858095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe34107710 msgr2=0x7efe3419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe31d9b640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe34107710 0x7efe3419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe31d9b640 1 -- 172.21.15.17:0/3966858095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe34069d50 msgr2=0x7efe34197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe31d9b640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe34069d50 0x7efe34197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe31d9b640 1 -- 172.21.15.17:0/3966858095 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3419cf00 con 0x7efe3406a710 2024-03-24T03:11:47.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe3259c640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe34069d50 0x7efe34197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:11:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe31d9b640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe3406a710 0x7efe34198450 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efe2800e9f0 tx=0x7efe2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe237fe640 1 -- 172.21.15.17:0/3966858095 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2800ccc0 con 0x7efe3406a710 2024-03-24T03:11:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe38d5b640 1 -- 172.21.15.17:0/3966858095 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3419d1f0 con 0x7efe3406a710 2024-03-24T03:11:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe237fe640 1 -- 172.21.15.17:0/3966858095 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe2800ce60 con 0x7efe3406a710 2024-03-24T03:11:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe237fe640 1 -- 172.21.15.17:0/3966858095 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28010690 con 0x7efe3406a710 2024-03-24T03:11:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.619+0000 7efe38d5b640 1 -- 172.21.15.17:0/3966858095 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe341a4a80 con 0x7efe3406a710 2024-03-24T03:11:47.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.620+0000 7efe38d5b640 1 -- 172.21.15.17:0/3966858095 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe34102d90 con 0x7efe3406a710 2024-03-24T03:11:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.621+0000 7efe237fe640 1 -- 172.21.15.17:0/3966858095 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7efe280046c0 con 0x7efe3406a710 2024-03-24T03:11:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.622+0000 7efe237fe640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7efe1c0a2970 0x7efe1c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.622+0000 7efe3259c640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7efe1c0a2970 0x7efe1c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.622+0000 7efe237fe640 1 -- 172.21.15.17:0/3966858095 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9708+0+0 (secure 0 0 0) 0x7efe280cf6c0 con 0x7efe3406a710 2024-03-24T03:11:47.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.622+0000 7efe3259c640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7efe1c0a2970 0x7efe1c0a4e30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009900 tx=0x7efe2c000bc0 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:47.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.626+0000 7efe237fe640 1 -- 172.21.15.17:0/3966858095 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efe2809bf80 con 0x7efe3406a710 2024-03-24T03:11:47.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:47 smithi188 ceph-mon[36206]: pgmap v333: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:47.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:47 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2374250535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:11:47.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:47 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2765713616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:11:47.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:47 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2595902226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:11:47.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.809+0000 7efe38d5b640 1 -- 172.21.15.17:0/3966858095 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe34199260 con 0x7efe3406a710 2024-03-24T03:11:47.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.809+0000 7efe237fe640 1 -- 172.21.15.17:0/3966858095 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe2809b880 con 0x7efe3406a710 2024-03-24T03:11:47.809 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-24T03:11:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.811+0000 7efe38d5b640 1 -- 172.21.15.17:0/3966858095 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7efe1c0a2970 msgr2=0x7efe1c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.811+0000 7efe38d5b640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7efe1c0a2970 0x7efe1c0a4e30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009900 tx=0x7efe2c000bc0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.811+0000 7efe38d5b640 1 -- 172.21.15.17:0/3966858095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe3406a710 msgr2=0x7efe34198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.811+0000 7efe38d5b640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe3406a710 0x7efe34198450 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efe2800e9f0 tx=0x7efe2800edf0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.811+0000 7efe38d5b640 1 -- 172.21.15.17:0/3966858095 shutdown_connections 2024-03-24T03:11:47.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.811+0000 7efe38d5b640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7efe1c0a2970 0x7efe1c0a4e30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.811+0000 7efe38d5b640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe34107710 0x7efe3419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.811+0000 7efe38d5b640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe3406a710 0x7efe34198450 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.811+0000 7efe38d5b640 1 --2- 172.21.15.17:0/3966858095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe34069d50 0x7efe34197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.811+0000 7efe38d5b640 1 -- 172.21.15.17:0/3966858095 >> 172.21.15.17:0/3966858095 conn(0x7efe34077670 msgr2=0x7efe340fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.811+0000 7efe38d5b640 1 -- 172.21.15.17:0/3966858095 shutdown_connections 2024-03-24T03:11:47.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.811+0000 7efe38d5b640 1 -- 172.21.15.17:0/3966858095 wait complete. 2024-03-24T03:11:47.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:47 smithi019 ceph-mon[35944]: pgmap v333: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:47.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:47 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2374250535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:11:47.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:47 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2765713616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:11:47.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:47 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2595902226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:11:47.821 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:47 smithi017 ceph-mon[31926]: pgmap v333: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-24T03:11:47.821 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:47 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2374250535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:11:47.821 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:47 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2765713616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T03:11:47.821 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:47 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2595902226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-24T03:11:47.822 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-24T03:11:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.929+0000 7f54f984b640 1 -- 172.21.15.17:0/3349118206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f41028e0 msgr2=0x7f54f4102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.929+0000 7f54f984b640 1 --2- 172.21.15.17:0/3349118206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f41028e0 0x7f54f4102cc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009900 tx=0x7f54e402bcd0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.929+0000 7f54f984b640 1 -- 172.21.15.17:0/3349118206 shutdown_connections 2024-03-24T03:11:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.929+0000 7f54f984b640 1 --2- 172.21.15.17:0/3349118206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f410bcb0 0x7f54f410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.929+0000 7f54f984b640 1 --2- 172.21.15.17:0/3349118206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f4103200 0x7f54f410b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.929+0000 7f54f984b640 1 --2- 172.21.15.17:0/3349118206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f41028e0 0x7f54f4102cc0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.929+0000 7f54f984b640 1 -- 172.21.15.17:0/3349118206 >> 172.21.15.17:0/3349118206 conn(0x7f54f4076920 msgr2=0x7f54f4076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.929+0000 7f54f984b640 1 -- 172.21.15.17:0/3349118206 shutdown_connections 2024-03-24T03:11:47.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.929+0000 7f54f984b640 1 -- 172.21.15.17:0/3349118206 wait complete. 2024-03-24T03:11:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.929+0000 7f54f984b640 1 Processor -- start 2024-03-24T03:11:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.929+0000 7f54f984b640 1 -- start start 2024-03-24T03:11:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.929+0000 7f54f984b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f41028e0 0x7f54f4195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f984b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f4103200 0x7f54f4196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f984b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f410bcb0 0x7f54f419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f984b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f410f830 con 0x7f54f410bcb0 2024-03-24T03:11:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f984b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f410f6b0 con 0x7f54f41028e0 2024-03-24T03:11:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f984b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f410f9b0 con 0x7f54f4103200 2024-03-24T03:11:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f410bcb0 0x7f54f419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f410bcb0 0x7f54f419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44152/0 (socket says 172.21.15.17:44152) 2024-03-24T03:11:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f37fe640 1 -- 172.21.15.17:0/247567004 learned_addr learned my addr 172.21.15.17:0/247567004 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f2ffd640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f41028e0 0x7f54f4195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f27fc640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f4103200 0x7f54f4196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f37fe640 1 -- 172.21.15.17:0/247567004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f4103200 msgr2=0x7f54f4196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f37fe640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f4103200 0x7f54f4196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f37fe640 1 -- 172.21.15.17:0/247567004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f41028e0 msgr2=0x7f54f4195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f37fe640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f41028e0 0x7f54f4195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f37fe640 1 -- 172.21.15.17:0/247567004 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f419ad40 con 0x7f54f410bcb0 2024-03-24T03:11:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f37fe640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f410bcb0 0x7f54f419a610 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009f30 tx=0x7f54ec012550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:47.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f8849640 1 -- 172.21.15.17:0/247567004 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec01c030 con 0x7f54f410bcb0 2024-03-24T03:11:47.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f8849640 1 -- 172.21.15.17:0/247567004 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54ec003e50 con 0x7f54f410bcb0 2024-03-24T03:11:47.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f984b640 1 -- 172.21.15.17:0/247567004 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f419b030 con 0x7f54f410bcb0 2024-03-24T03:11:47.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f8849640 1 -- 172.21.15.17:0/247567004 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec018660 con 0x7f54f410bcb0 2024-03-24T03:11:47.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.930+0000 7f54f984b640 1 -- 172.21.15.17:0/247567004 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f41a28c0 con 0x7f54f410bcb0 2024-03-24T03:11:47.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.931+0000 7f54f984b640 1 -- 172.21.15.17:0/247567004 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f406a490 con 0x7f54f410bcb0 2024-03-24T03:11:47.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.932+0000 7f54f8849640 1 -- 172.21.15.17:0/247567004 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f54ec01f070 con 0x7f54f410bcb0 2024-03-24T03:11:47.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.933+0000 7f54f8849640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f54dc0a2970 0x7f54dc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:47.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.933+0000 7f54f8849640 1 -- 172.21.15.17:0/247567004 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9708+0+0 (secure 0 0 0) 0x7f54ec0d2fe0 con 0x7f54f410bcb0 2024-03-24T03:11:47.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.933+0000 7f54f2ffd640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f54dc0a2970 0x7f54dc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:47.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.933+0000 7f54f2ffd640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f54dc0a2970 0x7f54dc0a4e30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54e402efd0 tx=0x7f54e402cf90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:47.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:47.937+0000 7f54f8849640 1 -- 172.21.15.17:0/247567004 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f54ec026070 con 0x7f54f410bcb0 2024-03-24T03:11:48.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.120+0000 7f54f984b640 1 -- 172.21.15.17:0/247567004 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f54f41970a0 con 0x7f54f410bcb0 2024-03-24T03:11:48.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.121+0000 7f54f8849640 1 -- 172.21.15.17:0/247567004 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v103) v1 ==== 103+0+11 (secure 0 0 0) 0x7f54ec09f840 con 0x7f54f410bcb0 2024-03-24T03:11:48.120 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-24T03:11:48.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.123+0000 7f54f984b640 1 -- 172.21.15.17:0/247567004 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f54dc0a2970 msgr2=0x7f54dc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:48.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.123+0000 7f54f984b640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f54dc0a2970 0x7f54dc0a4e30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54e402efd0 tx=0x7f54e402cf90 comp rx=0 tx=0).stop 2024-03-24T03:11:48.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.123+0000 7f54f984b640 1 -- 172.21.15.17:0/247567004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f410bcb0 msgr2=0x7f54f419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:48.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.123+0000 7f54f984b640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f410bcb0 0x7f54f419a610 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009f30 tx=0x7f54ec012550 comp rx=0 tx=0).stop 2024-03-24T03:11:48.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.123+0000 7f54f984b640 1 -- 172.21.15.17:0/247567004 shutdown_connections 2024-03-24T03:11:48.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.123+0000 7f54f984b640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f54dc0a2970 0x7f54dc0a4e30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.123+0000 7f54f984b640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54f410bcb0 0x7f54f419a610 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.123+0000 7f54f984b640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f4103200 0x7f54f4196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.123+0000 7f54f984b640 1 --2- 172.21.15.17:0/247567004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f41028e0 0x7f54f4195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.123+0000 7f54f984b640 1 -- 172.21.15.17:0/247567004 >> 172.21.15.17:0/247567004 conn(0x7f54f4076920 msgr2=0x7f54f40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:48.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.123+0000 7f54f984b640 1 -- 172.21.15.17:0/247567004 shutdown_connections 2024-03-24T03:11:48.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.123+0000 7f54f984b640 1 -- 172.21.15.17:0/247567004 wait complete. 2024-03-24T03:11:48.133 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-24T03:11:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.239+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/965293119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0100f80 msgr2=0x7fcdd010d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.239+0000 7fcdd4dc4640 1 --2- 172.21.15.17:0/965293119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0100f80 0x7fcdd010d490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009be0 tx=0x7fcdc802bef0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.239+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/965293119 shutdown_connections 2024-03-24T03:11:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.239+0000 7fcdd4dc4640 1 --2- 172.21.15.17:0/965293119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0100f80 0x7fcdd010d490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.239+0000 7fcdd4dc4640 1 --2- 172.21.15.17:0/965293119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdd01005c0 0x7fcdd0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.239+0000 7fcdd4dc4640 1 --2- 172.21.15.17:0/965293119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdd0106660 0x7fcdd0106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.239+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/965293119 >> 172.21.15.17:0/965293119 conn(0x7fcdd0077700 msgr2=0x7fcdd0077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.239+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/965293119 shutdown_connections 2024-03-24T03:11:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.239+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/965293119 wait complete. 2024-03-24T03:11:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdd4dc4640 1 Processor -- start 2024-03-24T03:11:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdd4dc4640 1 -- start start 2024-03-24T03:11:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdd4dc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdd01005c0 0x7fcdd0197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdd4dc4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdd0100f80 0x7fcdd01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdd4dc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0106660 0x7fcdd019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdd4dc4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0111af0 con 0x7fcdd0100f80 2024-03-24T03:11:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdd4dc4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0111970 con 0x7fcdd0106660 2024-03-24T03:11:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdd4dc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0111c70 con 0x7fcdd01005c0 2024-03-24T03:11:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdcdd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdd0100f80 0x7fcdd01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdce59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdd01005c0 0x7fcdd0197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdcdd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdd0100f80 0x7fcdd01984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44174/0 (socket says 172.21.15.17:44174) 2024-03-24T03:11:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdcdd9b640 1 -- 172.21.15.17:0/1371362297 learned_addr learned my addr 172.21.15.17:0/1371362297 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.240+0000 7fcdced9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0106660 0x7fcdd019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdcdd9b640 1 -- 172.21.15.17:0/1371362297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdd01005c0 msgr2=0x7fcdd0197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdcdd9b640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdd01005c0 0x7fcdd0197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdcdd9b640 1 -- 172.21.15.17:0/1371362297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0106660 msgr2=0x7fcdd019c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdcdd9b640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0106660 0x7fcdd019c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdcdd9b640 1 -- 172.21.15.17:0/1371362297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd019cf60 con 0x7fcdd0100f80 2024-03-24T03:11:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdce59c640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdd01005c0 0x7fcdd0197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:11:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdced9d640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0106660 0x7fcdd019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:11:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdcdd9b640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdd0100f80 0x7fcdd01984b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400e9f0 tx=0x7fcdc400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdbf7fe640 1 -- 172.21.15.17:0/1371362297 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc400cd80 con 0x7fcdd0100f80 2024-03-24T03:11:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdbf7fe640 1 -- 172.21.15.17:0/1371362297 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdc4005500 con 0x7fcdd0100f80 2024-03-24T03:11:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/1371362297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd019d250 con 0x7fcdd0100f80 2024-03-24T03:11:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdbf7fe640 1 -- 172.21.15.17:0/1371362297 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4010690 con 0x7fcdd0100f80 2024-03-24T03:11:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.241+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/1371362297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd01a4ae0 con 0x7fcdd0100f80 2024-03-24T03:11:48.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.242+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/1371362297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd00688e0 con 0x7fcdd0100f80 2024-03-24T03:11:48.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.244+0000 7fcdbf7fe640 1 -- 172.21.15.17:0/1371362297 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fcdc40046c0 con 0x7fcdd0100f80 2024-03-24T03:11:48.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.244+0000 7fcdbf7fe640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fcdb80a2970 0x7fcdb80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:48.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.244+0000 7fcdce59c640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fcdb80a2970 0x7fcdb80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:48.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.244+0000 7fcdbf7fe640 1 -- 172.21.15.17:0/1371362297 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9708+0+0 (secure 0 0 0) 0x7fcdc40cf940 con 0x7fcdd0100f80 2024-03-24T03:11:48.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.245+0000 7fcdce59c640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fcdb80a2970 0x7fcdb80a4e30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00096c0 tx=0x7fcdc0009260 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:48.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.248+0000 7fcdbf7fe640 1 -- 172.21.15.17:0/1371362297 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcdc4053740 con 0x7fcdd0100f80 2024-03-24T03:11:48.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.443+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/1371362297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcdd01992c0 con 0x7fcdd0100f80 2024-03-24T03:11:48.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.444+0000 7fcdbf7fe640 1 -- 172.21.15.17:0/1371362297 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v103) v1 ==== 81+0+997 (secure 0 0 0) 0x7fcdc409c1a0 con 0x7fcdd0100f80 2024-03-24T03:11:48.443 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/467902990 2024-03-25T02:51:29.864563+0000 2024-03-24T03:11:48.443 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/532405 2024-03-25T02:51:29.864563+0000 2024-03-24T03:11:48.443 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/934316425 2024-03-25T03:01:04.109518+0000 2024-03-24T03:11:48.443 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/2663792883 2024-03-25T03:01:04.109518+0000 2024-03-24T03:11:48.444 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/4202143244 2024-03-25T02:51:02.292414+0000 2024-03-24T03:11:48.444 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/4291784529 2024-03-25T02:51:02.292414+0000 2024-03-24T03:11:48.444 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/2226886960 2024-03-25T02:51:02.292414+0000 2024-03-24T03:11:48.444 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/1698088665 2024-03-25T03:01:04.109518+0000 2024-03-24T03:11:48.444 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/3007657427 2024-03-25T02:51:29.864563+0000 2024-03-24T03:11:48.444 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/3007657427 2024-03-25T02:51:29.864563+0000 2024-03-24T03:11:48.444 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/1870031079 2024-03-25T02:51:29.864563+0000 2024-03-24T03:11:48.444 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/2663792883 2024-03-25T03:01:04.109518+0000 2024-03-24T03:11:48.444 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/630595937 2024-03-25T02:51:02.292414+0000 2024-03-24T03:11:48.444 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/4291784529 2024-03-25T02:51:02.292414+0000 2024-03-24T03:11:48.444 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3359268740 2024-03-25T03:01:04.109518+0000 2024-03-24T03:11:48.444 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/1112257866 2024-03-24T04:05:56.644241+0000 2024-03-24T03:11:48.445 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/1127409383 2024-03-25T03:01:04.109518+0000 2024-03-24T03:11:48.445 INFO:teuthology.orchestra.run.smithi017.stderr:listed 17 entries 2024-03-24T03:11:48.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.445+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/1371362297 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fcdb80a2970 msgr2=0x7fcdb80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:48.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.445+0000 7fcdd4dc4640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fcdb80a2970 0x7fcdb80a4e30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00096c0 tx=0x7fcdc0009260 comp rx=0 tx=0).stop 2024-03-24T03:11:48.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.445+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/1371362297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdd0100f80 msgr2=0x7fcdd01984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:48.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.445+0000 7fcdd4dc4640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdd0100f80 0x7fcdd01984b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400e9f0 tx=0x7fcdc400edf0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.445+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/1371362297 shutdown_connections 2024-03-24T03:11:48.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.445+0000 7fcdd4dc4640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fcdb80a2970 0x7fcdb80a4e30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.445+0000 7fcdd4dc4640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0106660 0x7fcdd019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.446+0000 7fcdd4dc4640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdd0100f80 0x7fcdd01984b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.446+0000 7fcdd4dc4640 1 --2- 172.21.15.17:0/1371362297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdd01005c0 0x7fcdd0197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.446+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/1371362297 >> 172.21.15.17:0/1371362297 conn(0x7fcdd0077700 msgr2=0x7fcdd00ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:48.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.446+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/1371362297 shutdown_connections 2024-03-24T03:11:48.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.446+0000 7fcdd4dc4640 1 -- 172.21.15.17:0/1371362297 wait complete. 2024-03-24T03:11:48.456 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-24T03:11:48.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:11:48.456 DEBUG:teuthology.orchestra.run.smithi017:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-24T03:11:48.505 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi017.front.sepia.ceph.com 2024-03-24T03:11:48.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:11:48.506 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-24T03:11:48.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:11:48.529 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-03-24T03:11:48.585 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-24T03:11:48.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:11:48.586 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-24T03:11:48.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T03:11:48.653 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link delete ceph-brx 2024-03-24T03:11:48.705 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:48 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3966858095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:11:48.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:48 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/247567004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:11:48.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:48 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/1371362297' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T03:11:48.755 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-24T03:11:48.766 INFO:tasks.cephadm:Teardown begin 2024-03-24T03:11:48.766 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-24T03:11:48.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T03:11:48.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:48 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3966858095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:11:48.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:48 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/247567004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:11:48.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:48 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/1371362297' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T03:11:48.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.900+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/210751984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e4100f40 msgr2=0x7fb3e410d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:48.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.900+0000 7fb3ec2dc640 1 --2- 172.21.15.17:0/210751984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e4100f40 0x7fb3e410d400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009cd0 tx=0x7fb3dc02c0f0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.900+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/210751984 shutdown_connections 2024-03-24T03:11:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.900+0000 7fb3ec2dc640 1 --2- 172.21.15.17:0/210751984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e4100f40 0x7fb3e410d400 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.900+0000 7fb3ec2dc640 1 --2- 172.21.15.17:0/210751984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e4100580 0x7fb3e4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.900+0000 7fb3ec2dc640 1 --2- 172.21.15.17:0/210751984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e4106640 0x7fb3e4106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.900+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/210751984 >> 172.21.15.17:0/210751984 conn(0x7fb3e4077640 msgr2=0x7fb3e4077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.900+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/210751984 shutdown_connections 2024-03-24T03:11:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.900+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/210751984 wait complete. 2024-03-24T03:11:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3ec2dc640 1 Processor -- start 2024-03-24T03:11:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3ec2dc640 1 -- start start 2024-03-24T03:11:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3ec2dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e4100580 0x7fb3e4196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3ec2dc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e4100f40 0x7fb3e4197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3ec2dc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e4106640 0x7fb3e419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3ec2dc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4110920 con 0x7fb3e4100f40 2024-03-24T03:11:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3ec2dc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e41107a0 con 0x7fb3e4100580 2024-03-24T03:11:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3ec2dc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4110aa0 con 0x7fb3e4106640 2024-03-24T03:11:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3e9877640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e4100f40 0x7fb3e4197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3e9877640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e4100f40 0x7fb3e4197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44184/0 (socket says 172.21.15.17:44184) 2024-03-24T03:11:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3e9877640 1 -- 172.21.15.17:0/38562720 learned_addr learned my addr 172.21.15.17:0/38562720 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3ea078640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e4100580 0x7fb3e4196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3ea879640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e4106640 0x7fb3e419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3e9877640 1 -- 172.21.15.17:0/38562720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e4106640 msgr2=0x7fb3e419b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3e9877640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e4106640 0x7fb3e419b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3e9877640 1 -- 172.21.15.17:0/38562720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e4100580 msgr2=0x7fb3e4196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3e9877640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e4100580 0x7fb3e4196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3e9877640 1 -- 172.21.15.17:0/38562720 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e419bd30 con 0x7fb3e4100f40 2024-03-24T03:11:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.901+0000 7fb3ea078640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e4100580 0x7fb3e4196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:11:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.902+0000 7fb3e9877640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e4100f40 0x7fb3e4197280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800e9f0 tx=0x7fb3d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.902+0000 7fb3d77fe640 1 -- 172.21.15.17:0/38562720 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d800cd80 con 0x7fb3e4100f40 2024-03-24T03:11:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.902+0000 7fb3d77fe640 1 -- 172.21.15.17:0/38562720 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3d8005500 con 0x7fb3e4100f40 2024-03-24T03:11:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.902+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/38562720 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e419c020 con 0x7fb3e4100f40 2024-03-24T03:11:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.902+0000 7fb3d77fe640 1 -- 172.21.15.17:0/38562720 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8010690 con 0x7fb3e4100f40 2024-03-24T03:11:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.902+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/38562720 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e41a38b0 con 0x7fb3e4100f40 2024-03-24T03:11:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.903+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/38562720 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e40688e0 con 0x7fb3e4100f40 2024-03-24T03:11:48.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.905+0000 7fb3d77fe640 1 -- 172.21.15.17:0/38562720 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb3d80046c0 con 0x7fb3e4100f40 2024-03-24T03:11:48.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.905+0000 7fb3d77fe640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb3d00a2970 0x7fb3d00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:48.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.905+0000 7fb3ea078640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb3d00a2970 0x7fb3d00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:48.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.905+0000 7fb3d77fe640 1 -- 172.21.15.17:0/38562720 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9708+0+0 (secure 0 0 0) 0x7fb3d80cf8e0 con 0x7fb3e4100f40 2024-03-24T03:11:48.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.906+0000 7fb3ea078640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb3d00a2970 0x7fb3d00a4e30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00095f0 tx=0x7fb3e0005f90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:48.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:48.909+0000 7fb3d77fe640 1 -- 172.21.15.17:0/38562720 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb3d8053740 con 0x7fb3e4100f40 2024-03-24T03:11:49.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:48 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3966858095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-24T03:11:49.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:48 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/247567004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-24T03:11:49.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:48 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/1371362297' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T03:11:49.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.136+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/38562720 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3e4198090 con 0x7fb3e4100f40 2024-03-24T03:11:49.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.136+0000 7fb3d77fe640 1 -- 172.21.15.17:0/38562720 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11452 (secure 0 0 0) 0x7fb3e4198090 con 0x7fb3e4100f40 2024-03-24T03:11:49.136 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:11:49.136 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T02:58:36.476051+0000","modified":"2024-03-24T03:00:54.990766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24529,"mds_1":24521,"mds_2":24511,"mds_3":24493,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6841/1592961709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":1592961709},{"type":"v1","addr":"172.21.15.17:6841","nonce":1592961709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6839/2102503456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2102503456},{"type":"v1","addr":"172.21.15.19:6839","nonce":2102503456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24521,24493,24485,24511,24529]},"id":1}]} 2024-03-24T03:11:49.136 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 29 2024-03-24T03:11:49.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.139+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/38562720 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb3d00a2970 msgr2=0x7fb3d00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.139+0000 7fb3ec2dc640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb3d00a2970 0x7fb3d00a4e30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00095f0 tx=0x7fb3e0005f90 comp rx=0 tx=0).stop 2024-03-24T03:11:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.139+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/38562720 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e4100f40 msgr2=0x7fb3e4197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.139+0000 7fb3ec2dc640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e4100f40 0x7fb3e4197280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800e9f0 tx=0x7fb3d800edf0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.139+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/38562720 shutdown_connections 2024-03-24T03:11:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.139+0000 7fb3ec2dc640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb3d00a2970 0x7fb3d00a4e30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.139+0000 7fb3ec2dc640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e4106640 0x7fb3e419b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.139+0000 7fb3ec2dc640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3e4100f40 0x7fb3e4197280 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.139+0000 7fb3ec2dc640 1 --2- 172.21.15.17:0/38562720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e4100580 0x7fb3e4196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.139+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/38562720 >> 172.21.15.17:0/38562720 conn(0x7fb3e4077640 msgr2=0x7fb3e40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.139+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/38562720 shutdown_connections 2024-03-24T03:11:49.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.139+0000 7fb3ec2dc640 1 -- 172.21.15.17:0/38562720 wait complete. 2024-03-24T03:11:49.149 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T03:11:49.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.253+0000 7f9053e3f640 1 -- 172.21.15.17:0/3875100152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f904c106a60 msgr2=0x7f904c10e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.253+0000 7f9053e3f640 1 --2- 172.21.15.17:0/3875100152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f904c106a60 0x7f904c10e670 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9044009ca0 tx=0x7f904402c0f0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.253+0000 7f9053e3f640 1 -- 172.21.15.17:0/3875100152 shutdown_connections 2024-03-24T03:11:49.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.253+0000 7f9053e3f640 1 --2- 172.21.15.17:0/3875100152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f904c106a60 0x7f904c10e670 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.253+0000 7f9053e3f640 1 --2- 172.21.15.17:0/3875100152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f904c100c50 0x7f904c106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.253+0000 7f9053e3f640 1 --2- 172.21.15.17:0/3875100152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f904c1002a0 0x7f904c100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.253+0000 7f9053e3f640 1 -- 172.21.15.17:0/3875100152 >> 172.21.15.17:0/3875100152 conn(0x7f904c0758b0 msgr2=0x7f904c075cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.253+0000 7f9053e3f640 1 -- 172.21.15.17:0/3875100152 shutdown_connections 2024-03-24T03:11:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.253+0000 7f9053e3f640 1 -- 172.21.15.17:0/3875100152 wait complete. 2024-03-24T03:11:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f9053e3f640 1 Processor -- start 2024-03-24T03:11:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f9053e3f640 1 -- start start 2024-03-24T03:11:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f9053e3f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f904c1002a0 0x7f904c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f9053e3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f904c106a60 0x7f904c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f9053e3f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f904c19c920 0x7f904c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f9053e3f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c111b50 con 0x7f904c19c920 2024-03-24T03:11:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f9053e3f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c1119d0 con 0x7f904c106a60 2024-03-24T03:11:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f9053e3f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c111cd0 con 0x7f904c1002a0 2024-03-24T03:11:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f90523dc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f904c19c920 0x7f904c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f90513da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f904c106a60 0x7f904c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f90523dc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f904c19c920 0x7f904c19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44196/0 (socket says 172.21.15.17:44196) 2024-03-24T03:11:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f90523dc640 1 -- 172.21.15.17:0/3485723015 learned_addr learned my addr 172.21.15.17:0/3485723015 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f9051bdb640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f904c1002a0 0x7f904c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f90523dc640 1 -- 172.21.15.17:0/3485723015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f904c1002a0 msgr2=0x7f904c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f90523dc640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f904c1002a0 0x7f904c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f90523dc640 1 -- 172.21.15.17:0/3485723015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f904c106a60 msgr2=0x7f904c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f90523dc640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f904c106a60 0x7f904c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f90523dc640 1 -- 172.21.15.17:0/3485723015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c19d410 con 0x7f904c19c920 2024-03-24T03:11:49.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.254+0000 7f90513da640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f904c106a60 0x7f904c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:11:49.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.255+0000 7f90523dc640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f904c19c920 0x7f904c19cdd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f904402c700 tx=0x7f9044032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:49.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.255+0000 7f903effd640 1 -- 172.21.15.17:0/3485723015 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044041020 con 0x7f904c19c920 2024-03-24T03:11:49.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.255+0000 7f9053e3f640 1 -- 172.21.15.17:0/3485723015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904c19d6a0 con 0x7f904c19c920 2024-03-24T03:11:49.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.255+0000 7f903effd640 1 -- 172.21.15.17:0/3485723015 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9044034cd0 con 0x7f904c19c920 2024-03-24T03:11:49.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.255+0000 7f903effd640 1 -- 172.21.15.17:0/3485723015 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044039040 con 0x7f904c19c920 2024-03-24T03:11:49.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.255+0000 7f9053e3f640 1 -- 172.21.15.17:0/3485723015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904c1a4e70 con 0x7f904c19c920 2024-03-24T03:11:49.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.256+0000 7f9053e3f640 1 -- 172.21.15.17:0/3485723015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904c0697d0 con 0x7f904c19c920 2024-03-24T03:11:49.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.257+0000 7f903effd640 1 -- 172.21.15.17:0/3485723015 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9044044020 con 0x7f904c19c920 2024-03-24T03:11:49.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.257+0000 7f903effd640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f90380a2970 0x7f90380a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:49.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.257+0000 7f9051bdb640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f90380a2970 0x7f90380a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:49.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.258+0000 7f903effd640 1 -- 172.21.15.17:0/3485723015 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9708+0+0 (secure 0 0 0) 0x7f90440f0e80 con 0x7f904c19c920 2024-03-24T03:11:49.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.258+0000 7f9051bdb640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f90380a2970 0x7f90380a4e30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9048009bd0 tx=0x7f9048005f90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:49.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.262+0000 7f903effd640 1 -- 172.21.15.17:0/3485723015 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f904404b020 con 0x7f904c19c920 2024-03-24T03:11:49.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.485+0000 7f9053e3f640 1 -- 172.21.15.17:0/3485723015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f904c109d60 con 0x7f904c19c920 2024-03-24T03:11:49.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.485+0000 7f903effd640 1 -- 172.21.15.17:0/3485723015 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11452 (secure 0 0 0) 0x7f90440bd850 con 0x7f904c19c920 2024-03-24T03:11:49.485 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:11:49.485 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2854025639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2854025639},{"type":"v1","addr":"172.21.15.17:6835","nonce":2854025639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/3530669946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":3530669946},{"type":"v1","addr":"172.21.15.17:6837","nonce":3530669946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T02:58:36.476051+0000","modified":"2024-03-24T03:00:54.990766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24529,"mds_1":24521,"mds_2":24511,"mds_3":24493,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6839/2895019504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2895019504},{"type":"v1","addr":"172.21.15.17:6839","nonce":2895019504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6841/1592961709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":1592961709},{"type":"v1","addr":"172.21.15.17:6841","nonce":1592961709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6833/3469034636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3469034636},{"type":"v1","addr":"172.21.15.19:6833","nonce":3469034636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/920982807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":920982807},{"type":"v1","addr":"172.21.15.188:6833","nonce":920982807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/1025006466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1025006466},{"type":"v1","addr":"172.21.15.19:6835","nonce":1025006466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6837/2540260543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2540260543},{"type":"v1","addr":"172.21.15.19:6837","nonce":2540260543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.188:6835/3649219818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3649219818},{"type":"v1","addr":"172.21.15.188:6835","nonce":3649219818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/3512350624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3512350624},{"type":"v1","addr":"172.21.15.188:6837","nonce":3512350624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6839/2102503456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2102503456},{"type":"v1","addr":"172.21.15.19:6839","nonce":2102503456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.188:6839/81409664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":81409664},{"type":"v1","addr":"172.21.15.188:6839","nonce":81409664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24521,24493,24485,24511,24529]},"id":1}]} 2024-03-24T03:11:49.485 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 29 2024-03-24T03:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.488+0000 7f9053e3f640 1 -- 172.21.15.17:0/3485723015 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f90380a2970 msgr2=0x7f90380a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.488+0000 7f9053e3f640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f90380a2970 0x7f90380a4e30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9048009bd0 tx=0x7f9048005f90 comp rx=0 tx=0).stop 2024-03-24T03:11:49.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.488+0000 7f9053e3f640 1 -- 172.21.15.17:0/3485723015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f904c19c920 msgr2=0x7f904c19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.488+0000 7f9053e3f640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f904c19c920 0x7f904c19cdd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f904402c700 tx=0x7f9044032f20 comp rx=0 tx=0).stop 2024-03-24T03:11:49.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.488+0000 7f9053e3f640 1 -- 172.21.15.17:0/3485723015 shutdown_connections 2024-03-24T03:11:49.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.488+0000 7f9053e3f640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f90380a2970 0x7f90380a4e30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.488+0000 7f9053e3f640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f904c19c920 0x7f904c19cdd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.488+0000 7f9053e3f640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f904c106a60 0x7f904c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.488+0000 7f9053e3f640 1 --2- 172.21.15.17:0/3485723015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f904c1002a0 0x7f904c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.488+0000 7f9053e3f640 1 -- 172.21.15.17:0/3485723015 >> 172.21.15.17:0/3485723015 conn(0x7f904c0758b0 msgr2=0x7f904c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:49.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.488+0000 7f9053e3f640 1 -- 172.21.15.17:0/3485723015 shutdown_connections 2024-03-24T03:11:49.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.488+0000 7f9053e3f640 1 -- 172.21.15.17:0/3485723015 wait complete. 2024-03-24T03:11:49.500 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T03:11:49.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.604+0000 7f6554976640 1 -- 172.21.15.17:0/995953408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6550106620 msgr2=0x7f6550106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.604+0000 7f6554976640 1 --2- 172.21.15.17:0/995953408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6550106620 0x7f6550106a00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6540009900 tx=0x7f654002bc30 comp rx=0 tx=0).stop 2024-03-24T03:11:49.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.604+0000 7f6554976640 1 -- 172.21.15.17:0/995953408 shutdown_connections 2024-03-24T03:11:49.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.604+0000 7f6554976640 1 --2- 172.21.15.17:0/995953408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6550100f20 0x7f655010d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.604+0000 7f6554976640 1 --2- 172.21.15.17:0/995953408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6550100560 0x7f65501009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.604+0000 7f6554976640 1 --2- 172.21.15.17:0/995953408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6550106620 0x7f6550106a00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.604+0000 7f6554976640 1 -- 172.21.15.17:0/995953408 >> 172.21.15.17:0/995953408 conn(0x7f6550076920 msgr2=0x7f6550076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:49.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.604+0000 7f6554976640 1 -- 172.21.15.17:0/995953408 shutdown_connections 2024-03-24T03:11:49.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.604+0000 7f6554976640 1 -- 172.21.15.17:0/995953408 wait complete. 2024-03-24T03:11:49.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f6554976640 1 Processor -- start 2024-03-24T03:11:49.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f6554976640 1 -- start start 2024-03-24T03:11:49.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f6554976640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6550100560 0x7f6550197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:49.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f6554976640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6550100f20 0x7f65501983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:49.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f6554976640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6550106620 0x7f655019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:49.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f6554976640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6550111ab0 con 0x7f6550106620 2024-03-24T03:11:49.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f6554976640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6550111930 con 0x7f6550100560 2024-03-24T03:11:49.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f6554976640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6550111c30 con 0x7f6550100f20 2024-03-24T03:11:49.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f654ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6550106620 0x7f655019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:49.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f654e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6550100560 0x7f6550197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:49.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f654ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6550106620 0x7f655019c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44208/0 (socket says 172.21.15.17:44208) 2024-03-24T03:11:49.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f654ed9d640 1 -- 172.21.15.17:0/2500974228 learned_addr learned my addr 172.21.15.17:0/2500974228 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:49.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.605+0000 7f6547fff640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6550100f20 0x7f65501983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:49.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.606+0000 7f654ed9d640 1 -- 172.21.15.17:0/2500974228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6550100f20 msgr2=0x7f65501983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.606+0000 7f654ed9d640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6550100f20 0x7f65501983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.606+0000 7f654ed9d640 1 -- 172.21.15.17:0/2500974228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6550100560 msgr2=0x7f6550197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.606+0000 7f654ed9d640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6550100560 0x7f6550197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.606+0000 7f654ed9d640 1 -- 172.21.15.17:0/2500974228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655019ce60 con 0x7f6550106620 2024-03-24T03:11:49.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.606+0000 7f654ed9d640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6550106620 0x7f655019c730 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f654800eb10 tx=0x7f654800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:49.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.606+0000 7f65477fe640 1 -- 172.21.15.17:0/2500974228 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548004b20 con 0x7f6550106620 2024-03-24T03:11:49.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.606+0000 7f65477fe640 1 -- 172.21.15.17:0/2500974228 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6548004cc0 con 0x7f6550106620 2024-03-24T03:11:49.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.606+0000 7f6554976640 1 -- 172.21.15.17:0/2500974228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655019d150 con 0x7f6550106620 2024-03-24T03:11:49.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.606+0000 7f65477fe640 1 -- 172.21.15.17:0/2500974228 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548010760 con 0x7f6550106620 2024-03-24T03:11:49.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.606+0000 7f6554976640 1 -- 172.21.15.17:0/2500974228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65501a49e0 con 0x7f6550106620 2024-03-24T03:11:49.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.607+0000 7f6554976640 1 -- 172.21.15.17:0/2500974228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6550101c80 con 0x7f6550106620 2024-03-24T03:11:49.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.608+0000 7f65477fe640 1 -- 172.21.15.17:0/2500974228 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6548010900 con 0x7f6550106620 2024-03-24T03:11:49.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.609+0000 7f65477fe640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f653c0a2970 0x7f653c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:49.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.609+0000 7f654e59c640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f653c0a2970 0x7f653c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:49.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.609+0000 7f65477fe640 1 -- 172.21.15.17:0/2500974228 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9708+0+0 (secure 0 0 0) 0x7f65480cfab0 con 0x7f6550106620 2024-03-24T03:11:49.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.610+0000 7f654e59c640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f653c0a2970 0x7f653c0a4e30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f654002efd0 tx=0x7f6540000bc0 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:49.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.613+0000 7f65477fe640 1 -- 172.21.15.17:0/2500974228 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6548053740 con 0x7f6550106620 2024-03-24T03:11:49.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.798+0000 7f6554976640 1 -- 172.21.15.17:0/2500974228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65501991c0 con 0x7f6550106620 2024-03-24T03:11:49.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.799+0000 7f65477fe640 1 -- 172.21.15.17:0/2500974228 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+25580 (secure 0 0 0) 0x7f6548086020 con 0x7f6550106620 2024-03-24T03:11:49.799 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-24T03:11:49.800 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":103,"fsid":"2a682800-e989-11ee-95cd-87774f69a715","created":"2024-03-24T02:50:35.152661+0000","modified":"2024-03-24T03:11:15.563541+0000","last_up_change":"2024-03-24T02:57:43.735664+0000","last_in_change":"2024-03-24T02:57:22.845804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T02:53:54.207139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-24T02:58:34.785192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-24T02:58:35.679772+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-24T02:58:39.221772+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07cdbc7a-3ba5-4284-8819-b9352e15e8c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6803","nonce":3184461551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6805","nonce":3184461551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6809","nonce":3184461551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3184461551},{"type":"v1","addr":"172.21.15.17:6807","nonce":3184461551}]},"public_addr":"172.21.15.17:6803/3184461551","cluster_addr":"172.21.15.17:6805/3184461551","heartbeat_back_addr":"172.21.15.17:6809/3184461551","heartbeat_front_addr":"172.21.15.17:6807/3184461551","state":["exists","up"]},{"osd":1,"uuid":"83235b08-6883-4443-8f53-7cd7f3db924f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6801","nonce":1476118517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6803","nonce":1476118517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6807","nonce":1476118517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1476118517},{"type":"v1","addr":"172.21.15.19:6805","nonce":1476118517}]},"public_addr":"172.21.15.19:6801/1476118517","cluster_addr":"172.21.15.19:6803/1476118517","heartbeat_back_addr":"172.21.15.19:6807/1476118517","heartbeat_front_addr":"172.21.15.19:6805/1476118517","state":["exists","up"]},{"osd":2,"uuid":"9e3c93f8-b9a9-4b21-ba59-6e3cd7736ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6801","nonce":2030207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6803","nonce":2030207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6807","nonce":2030207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2030207482},{"type":"v1","addr":"172.21.15.188:6805","nonce":2030207482}]},"public_addr":"172.21.15.188:6801/2030207482","cluster_addr":"172.21.15.188:6803/2030207482","heartbeat_back_addr":"172.21.15.188:6807/2030207482","heartbeat_front_addr":"172.21.15.188:6805/2030207482","state":["exists","up"]},{"osd":3,"uuid":"1ba93700-990f-4c99-9d48-1adf99322cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6811","nonce":1744816113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6813","nonce":1744816113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6817","nonce":1744816113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1744816113},{"type":"v1","addr":"172.21.15.17:6815","nonce":1744816113}]},"public_addr":"172.21.15.17:6811/1744816113","cluster_addr":"172.21.15.17:6813/1744816113","heartbeat_back_addr":"172.21.15.17:6817/1744816113","heartbeat_front_addr":"172.21.15.17:6815/1744816113","state":["exists","up"]},{"osd":4,"uuid":"659ccfee-f6c0-4d35-b557-f7978d553346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6809","nonce":4029793962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6811","nonce":4029793962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6815","nonce":4029793962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4029793962},{"type":"v1","addr":"172.21.15.19:6813","nonce":4029793962}]},"public_addr":"172.21.15.19:6809/4029793962","cluster_addr":"172.21.15.19:6811/4029793962","heartbeat_back_addr":"172.21.15.19:6815/4029793962","heartbeat_front_addr":"172.21.15.19:6813/4029793962","state":["exists","up"]},{"osd":5,"uuid":"a3ec6587-2c61-4188-b4af-9f36ad6c4abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6809","nonce":2503049360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6811","nonce":2503049360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6815","nonce":2503049360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2503049360},{"type":"v1","addr":"172.21.15.188:6813","nonce":2503049360}]},"public_addr":"172.21.15.188:6809/2503049360","cluster_addr":"172.21.15.188:6811/2503049360","heartbeat_back_addr":"172.21.15.188:6815/2503049360","heartbeat_front_addr":"172.21.15.188:6813/2503049360","state":["exists","up"]},{"osd":6,"uuid":"6713217d-57aa-4293-9cb7-de935d0a8537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6819","nonce":3664721354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6821","nonce":3664721354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6825","nonce":3664721354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3664721354},{"type":"v1","addr":"172.21.15.17:6823","nonce":3664721354}]},"public_addr":"172.21.15.17:6819/3664721354","cluster_addr":"172.21.15.17:6821/3664721354","heartbeat_back_addr":"172.21.15.17:6825/3664721354","heartbeat_front_addr":"172.21.15.17:6823/3664721354","state":["exists","up"]},{"osd":7,"uuid":"4aa6635c-d640-46d9-9124-3e7afe24e65d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6817","nonce":2743063507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6819","nonce":2743063507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6823","nonce":2743063507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2743063507},{"type":"v1","addr":"172.21.15.19:6821","nonce":2743063507}]},"public_addr":"172.21.15.19:6817/2743063507","cluster_addr":"172.21.15.19:6819/2743063507","heartbeat_back_addr":"172.21.15.19:6823/2743063507","heartbeat_front_addr":"172.21.15.19:6821/2743063507","state":["exists","up"]},{"osd":8,"uuid":"9df9a933-c0be-4277-9837-95fb58c813d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6817","nonce":1144752279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6819","nonce":1144752279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6823","nonce":1144752279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1144752279},{"type":"v1","addr":"172.21.15.188:6821","nonce":1144752279}]},"public_addr":"172.21.15.188:6817/1144752279","cluster_addr":"172.21.15.188:6819/1144752279","heartbeat_back_addr":"172.21.15.188:6823/1144752279","heartbeat_front_addr":"172.21.15.188:6821/1144752279","state":["exists","up"]},{"osd":9,"uuid":"50a0ca53-b96d-47ed-84f6-b7fc6a75c23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6827","nonce":116991070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6829","nonce":116991070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6833","nonce":116991070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":116991070},{"type":"v1","addr":"172.21.15.17:6831","nonce":116991070}]},"public_addr":"172.21.15.17:6827/116991070","cluster_addr":"172.21.15.17:6829/116991070","heartbeat_back_addr":"172.21.15.17:6833/116991070","heartbeat_front_addr":"172.21.15.17:6831/116991070","state":["exists","up"]},{"osd":10,"uuid":"e702b87a-9535-42db-aa8f-58f35cecb5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6825","nonce":3678221892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6827","nonce":3678221892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6831","nonce":3678221892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3678221892},{"type":"v1","addr":"172.21.15.19:6829","nonce":3678221892}]},"public_addr":"172.21.15.19:6825/3678221892","cluster_addr":"172.21.15.19:6827/3678221892","heartbeat_back_addr":"172.21.15.19:6831/3678221892","heartbeat_front_addr":"172.21.15.19:6829/3678221892","state":["exists","up"]},{"osd":11,"uuid":"bb1fc489-8371-4d59-9086-2629adec0374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6825","nonce":2579389713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6827","nonce":2579389713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6831","nonce":2579389713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2579389713},{"type":"v1","addr":"172.21.15.188:6829","nonce":2579389713}]},"public_addr":"172.21.15.188:6825/2579389713","cluster_addr":"172.21.15.188:6827/2579389713","heartbeat_back_addr":"172.21.15.188:6831/2579389713","heartbeat_front_addr":"172.21.15.188:6829/2579389713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:03.727635+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:27.769884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:53:51.147237+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:16.176900+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:54:41.878571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:06.075577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:31.684649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:55:57.845160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:22.661715+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:56:48.962183+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:14.777779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T02:57:41.035571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":7,"to":6}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/467902990":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/532405":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/934316425":"2024-03-25T03:01:04.109518+0000","172.21.15.17:6800/2663792883":"2024-03-25T03:01:04.109518+0000","172.21.15.17:0/4202143244":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6801/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/2226886960":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/1698088665":"2024-03-25T03:01:04.109518+0000","172.21.15.17:6800/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/3007657427":"2024-03-25T02:51:29.864563+0000","172.21.15.17:0/1870031079":"2024-03-25T02:51:29.864563+0000","172.21.15.17:6801/2663792883":"2024-03-25T03:01:04.109518+0000","172.21.15.17:0/630595937":"2024-03-25T02:51:02.292414+0000","172.21.15.17:6800/4291784529":"2024-03-25T02:51:02.292414+0000","172.21.15.17:0/3359268740":"2024-03-25T03:01:04.109518+0000","172.21.15.17:0/1112257866":"2024-03-24T04:05:56.644241+0000","172.21.15.17:0/1127409383":"2024-03-25T03:01:04.109518+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T03:11:49.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.801+0000 7f6554976640 1 -- 172.21.15.17:0/2500974228 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f653c0a2970 msgr2=0x7f653c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.801+0000 7f6554976640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f653c0a2970 0x7f653c0a4e30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f654002efd0 tx=0x7f6540000bc0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.801+0000 7f6554976640 1 -- 172.21.15.17:0/2500974228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6550106620 msgr2=0x7f655019c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.801+0000 7f6554976640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6550106620 0x7f655019c730 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f654800eb10 tx=0x7f654800c2d0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.801+0000 7f6554976640 1 -- 172.21.15.17:0/2500974228 shutdown_connections 2024-03-24T03:11:49.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.801+0000 7f6554976640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f653c0a2970 0x7f653c0a4e30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.801+0000 7f6554976640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6550106620 0x7f655019c730 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.801+0000 7f6554976640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6550100f20 0x7f65501983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.801+0000 7f6554976640 1 --2- 172.21.15.17:0/2500974228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6550100560 0x7f6550197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.801+0000 7f6554976640 1 -- 172.21.15.17:0/2500974228 >> 172.21.15.17:0/2500974228 conn(0x7f6550076920 msgr2=0x7f65500fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:49.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.802+0000 7f6554976640 1 -- 172.21.15.17:0/2500974228 shutdown_connections 2024-03-24T03:11:49.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.802+0000 7f6554976640 1 -- 172.21.15.17:0/2500974228 wait complete. 2024-03-24T03:11:49.811 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:49 smithi017 ceph-mon[31926]: pgmap v334: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:49.811 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:49 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/38562720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:11:49.811 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:49 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3485723015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:11:49.813 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-24T03:11:49.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:49 smithi019 ceph-mon[35944]: pgmap v334: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:49.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:49 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/38562720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:11:49.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:49 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3485723015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:11:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.918+0000 7f8ee264b640 1 -- 172.21.15.17:0/611740761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc069c10 msgr2=0x7f8edc06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.918+0000 7f8ee264b640 1 --2- 172.21.15.17:0/611740761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc069c10 0x7f8edc06a050 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009930 tx=0x7f8ed002bc30 comp rx=0 tx=0).stop 2024-03-24T03:11:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.918+0000 7f8ee264b640 1 -- 172.21.15.17:0/611740761 shutdown_connections 2024-03-24T03:11:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.918+0000 7f8ee264b640 1 --2- 172.21.15.17:0/611740761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8edc06a590 0x7f8edc10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.918+0000 7f8ee264b640 1 --2- 172.21.15.17:0/611740761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc069c10 0x7f8edc06a050 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.918+0000 7f8ee264b640 1 --2- 172.21.15.17:0/611740761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8edc107660 0x7f8edc107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.918+0000 7f8ee264b640 1 -- 172.21.15.17:0/611740761 >> 172.21.15.17:0/611740761 conn(0x7f8edc077580 msgr2=0x7f8edc077990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:49.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.918+0000 7f8ee264b640 1 -- 172.21.15.17:0/611740761 shutdown_connections 2024-03-24T03:11:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.918+0000 7f8ee264b640 1 -- 172.21.15.17:0/611740761 wait complete. 2024-03-24T03:11:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.918+0000 7f8ee264b640 1 Processor -- start 2024-03-24T03:11:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.918+0000 7f8ee264b640 1 -- start start 2024-03-24T03:11:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee264b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8edc069c10 0x7f8edc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee264b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8edc06a590 0x7f8edc1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee264b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc107660 0x7f8edc19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee264b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc111a60 con 0x7f8edc107660 2024-03-24T03:11:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee264b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc1118e0 con 0x7f8edc069c10 2024-03-24T03:11:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee264b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc111be0 con 0x7f8edc06a590 2024-03-24T03:11:49.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee0be8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc107660 0x7f8edc19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee0be8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc107660 0x7f8edc19c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44232/0 (socket says 172.21.15.17:44232) 2024-03-24T03:11:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8edb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8edc06a590 0x7f8edc1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee0be8640 1 -- 172.21.15.17:0/2544453731 learned_addr learned my addr 172.21.15.17:0/2544453731 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8edbfff640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8edc069c10 0x7f8edc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee0be8640 1 -- 172.21.15.17:0/2544453731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8edc06a590 msgr2=0x7f8edc1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee0be8640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8edc06a590 0x7f8edc1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee0be8640 1 -- 172.21.15.17:0/2544453731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8edc069c10 msgr2=0x7f8edc197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee0be8640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8edc069c10 0x7f8edc197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee0be8640 1 -- 172.21.15.17:0/2544453731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc19cf70 con 0x7f8edc107660 2024-03-24T03:11:49.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee0be8640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc107660 0x7f8edc19c840 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400eb10 tx=0x7f8ed400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ed97fa640 1 -- 172.21.15.17:0/2544453731 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4004b20 con 0x7f8edc107660 2024-03-24T03:11:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ed97fa640 1 -- 172.21.15.17:0/2544453731 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ed4004cc0 con 0x7f8edc107660 2024-03-24T03:11:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee264b640 1 -- 172.21.15.17:0/2544453731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8edc19d260 con 0x7f8edc107660 2024-03-24T03:11:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ed97fa640 1 -- 172.21.15.17:0/2544453731 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4010760 con 0x7f8edc107660 2024-03-24T03:11:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.919+0000 7f8ee264b640 1 -- 172.21.15.17:0/2544453731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc1a4af0 con 0x7f8edc107660 2024-03-24T03:11:49.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.920+0000 7f8ee264b640 1 -- 172.21.15.17:0/2544453731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8edc102d90 con 0x7f8edc107660 2024-03-24T03:11:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.921+0000 7f8ed97fa640 1 -- 172.21.15.17:0/2544453731 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8ed4005020 con 0x7f8edc107660 2024-03-24T03:11:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.922+0000 7f8ed97fa640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f8ec40a2970 0x7f8ec40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:49.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.922+0000 7f8edbfff640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f8ec40a2970 0x7f8ec40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.922+0000 7f8ed97fa640 1 -- 172.21.15.17:0/2544453731 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9708+0+0 (secure 0 0 0) 0x7f8ed40cf880 con 0x7f8edc107660 2024-03-24T03:11:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.922+0000 7f8edbfff640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f8ec40a2970 0x7f8ec40a4e30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009b50 tx=0x7f8ecc005f90 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:49.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:49.926+0000 7f8ed97fa640 1 -- 172.21.15.17:0/2544453731 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8ed409c140 con 0x7f8edc107660 2024-03-24T03:11:50.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:49 smithi188 ceph-mon[36206]: pgmap v334: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-24T03:11:50.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:49 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/38562720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:11:50.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:49 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3485723015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T03:11:50.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.149+0000 7f8ee264b640 1 -- 172.21.15.17:0/2544453731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8edc1992d0 con 0x7f8edc107660 2024-03-24T03:11:50.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.151+0000 7f8ed97fa640 1 -- 172.21.15.17:0/2544453731 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8ed409ba40 con 0x7f8edc107660 2024-03-24T03:11:50.150 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-24T03:11:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.155+0000 7f8ee264b640 1 -- 172.21.15.17:0/2544453731 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f8ec40a2970 msgr2=0x7f8ec40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.155+0000 7f8ee264b640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f8ec40a2970 0x7f8ec40a4e30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009b50 tx=0x7f8ecc005f90 comp rx=0 tx=0).stop 2024-03-24T03:11:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.155+0000 7f8ee264b640 1 -- 172.21.15.17:0/2544453731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc107660 msgr2=0x7f8edc19c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.155+0000 7f8ee264b640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc107660 0x7f8edc19c840 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400eb10 tx=0x7f8ed400c2d0 comp rx=0 tx=0).stop 2024-03-24T03:11:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.156+0000 7f8ee264b640 1 -- 172.21.15.17:0/2544453731 shutdown_connections 2024-03-24T03:11:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.156+0000 7f8ee264b640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f8ec40a2970 0x7f8ec40a4e30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.156+0000 7f8ee264b640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc107660 0x7f8edc19c840 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.156+0000 7f8ee264b640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8edc06a590 0x7f8edc1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.156+0000 7f8ee264b640 1 --2- 172.21.15.17:0/2544453731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8edc069c10 0x7f8edc197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.156+0000 7f8ee264b640 1 -- 172.21.15.17:0/2544453731 >> 172.21.15.17:0/2544453731 conn(0x7f8edc077580 msgr2=0x7f8edc0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.156+0000 7f8ee264b640 1 -- 172.21.15.17:0/2544453731 shutdown_connections 2024-03-24T03:11:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.156+0000 7f8ee264b640 1 -- 172.21.15.17:0/2544453731 wait complete. 2024-03-24T03:11:50.169 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-24T03:11:50.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.285+0000 7f669564b640 1 -- 172.21.15.17:0/2029774211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690069260 msgr2=0x7f669010e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:50.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.285+0000 7f669564b640 1 --2- 172.21.15.17:0/2029774211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690069260 0x7f669010e500 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6688009cd0 tx=0x7f668802c0f0 comp rx=0 tx=0).stop 2024-03-24T03:11:50.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.285+0000 7f669564b640 1 -- 172.21.15.17:0/2029774211 shutdown_connections 2024-03-24T03:11:50.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.285+0000 7f669564b640 1 --2- 172.21.15.17:0/2029774211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690069260 0x7f669010e500 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.285+0000 7f669564b640 1 --2- 172.21.15.17:0/2029774211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66900688a0 0x7f6690068d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.285+0000 7f669564b640 1 --2- 172.21.15.17:0/2029774211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66901076f0 0x7f6690107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.285+0000 7f669564b640 1 -- 172.21.15.17:0/2029774211 >> 172.21.15.17:0/2029774211 conn(0x7f66900776a0 msgr2=0x7f6690077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.285+0000 7f669564b640 1 -- 172.21.15.17:0/2029774211 shutdown_connections 2024-03-24T03:11:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.285+0000 7f669564b640 1 -- 172.21.15.17:0/2029774211 wait complete. 2024-03-24T03:11:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f669564b640 1 Processor -- start 2024-03-24T03:11:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f669564b640 1 -- start start 2024-03-24T03:11:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f669564b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66900688a0 0x7f6690197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f669564b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690069260 0x7f6690198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f669564b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66901076f0 0x7f669019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f669564b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6690111a40 con 0x7f6690069260 2024-03-24T03:11:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f669564b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66901118c0 con 0x7f66901076f0 2024-03-24T03:11:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f669564b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6690111bc0 con 0x7f66900688a0 2024-03-24T03:11:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f6687fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690069260 0x7f6690198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f6687fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690069260 0x7f6690198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44286/0 (socket says 172.21.15.17:44286) 2024-03-24T03:11:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f6687fff640 1 -- 172.21.15.17:0/230532049 learned_addr learned my addr 172.21.15.17:0/230532049 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f668effd640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66900688a0 0x7f6690197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.286+0000 7f668f7fe640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66901076f0 0x7f669019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.287+0000 7f6687fff640 1 -- 172.21.15.17:0/230532049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66900688a0 msgr2=0x7f6690197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.287+0000 7f6687fff640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66900688a0 0x7f6690197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.287+0000 7f6687fff640 1 -- 172.21.15.17:0/230532049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66901076f0 msgr2=0x7f669019c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.287+0000 7f6687fff640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66901076f0 0x7f669019c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.287+0000 7f6687fff640 1 -- 172.21.15.17:0/230532049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669019cea0 con 0x7f6690069260 2024-03-24T03:11:50.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.287+0000 7f6687fff640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690069260 0x7f6690198440 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f667800e9f0 tx=0x7f667800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:50.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.287+0000 7f668cff9640 1 -- 172.21.15.17:0/230532049 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667800cd80 con 0x7f6690069260 2024-03-24T03:11:50.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.287+0000 7f668cff9640 1 -- 172.21.15.17:0/230532049 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6678005500 con 0x7f6690069260 2024-03-24T03:11:50.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.287+0000 7f669564b640 1 -- 172.21.15.17:0/230532049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669019d190 con 0x7f6690069260 2024-03-24T03:11:50.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.287+0000 7f668cff9640 1 -- 172.21.15.17:0/230532049 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678010690 con 0x7f6690069260 2024-03-24T03:11:50.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.287+0000 7f669564b640 1 -- 172.21.15.17:0/230532049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66901a4a20 con 0x7f6690069260 2024-03-24T03:11:50.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.288+0000 7f669564b640 1 -- 172.21.15.17:0/230532049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6690068490 con 0x7f6690069260 2024-03-24T03:11:50.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.289+0000 7f668cff9640 1 -- 172.21.15.17:0/230532049 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f66780046c0 con 0x7f6690069260 2024-03-24T03:11:50.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.290+0000 7f668cff9640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f667c0a2970 0x7f667c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:50.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.290+0000 7f668effd640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f667c0a2970 0x7f667c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:50.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.290+0000 7f668cff9640 1 -- 172.21.15.17:0/230532049 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 10050+0+0 (secure 0 0 0) 0x7f66780cfaa0 con 0x7f6690069260 2024-03-24T03:11:50.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.290+0000 7f668effd640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f667c0a2970 0x7f667c0a4e30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f668000bbe0 tx=0x7f668000b390 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:50.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.293+0000 7f668cff9640 1 -- 172.21.15.17:0/230532049 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6678053740 con 0x7f6690069260 2024-03-24T03:11:50.378 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a[31922]: 2024-03-24T03:11:50.150+0000 7f77d7e55700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T03:11:50.379 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-a[73909]: ignoring --setuser ceph since I am not root 2024-03-24T03:11:50.379 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-a[73909]: ignoring --setgroup ceph since I am not root 2024-03-24T03:11:50.379 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mds[73913]: -- 172.21.15.17:0/3570734881 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564d0b191a40 con 0x564d0b164800 2024-03-24T03:11:50.379 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-a[73909]: starting mds.a at 2024-03-24T03:11:50.379 INFO:journalctl@ceph.mds.g.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-g[73549]: ignoring --setuser ceph since I am not root 2024-03-24T03:11:50.379 INFO:journalctl@ceph.mds.g.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-g[73549]: ignoring --setgroup ceph since I am not root 2024-03-24T03:11:50.379 INFO:journalctl@ceph.mds.g.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mds[73553]: -- 172.21.15.17:0/3505674232 <== mon.0 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56068b795a40 con 0x56068b769000 2024-03-24T03:11:50.380 INFO:journalctl@ceph.mds.g.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-g[73549]: starting mds.g at 2024-03-24T03:11:50.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:50.519+0000 7f669564b640 1 -- 172.21.15.17:0/230532049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f669002d070 con 0x7f6690069260 2024-03-24T03:11:50.548 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-c[66846]: ignoring --setuser ceph since I am not root 2024-03-24T03:11:50.548 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-c[66846]: ignoring --setgroup ceph since I am not root 2024-03-24T03:11:50.548 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mds[66852]: -- 172.21.15.188:0/1015808196 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a363fc24e0 con 0x55a363f98800 2024-03-24T03:11:50.548 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mds[66852]: -- 172.21.15.188:0/1015808196 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a363fc5a40 con 0x55a363f98800 2024-03-24T03:11:50.548 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-c[66846]: starting mds.c at 2024-03-24T03:11:50.548 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-f[66535]: ignoring --setuser ceph since I am not root 2024-03-24T03:11:50.548 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-f[66535]: ignoring --setgroup ceph since I am not root 2024-03-24T03:11:50.548 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mds[66539]: -- 172.21.15.188:0/162153832 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fcd7251a40 con 0x55fcd7224800 2024-03-24T03:11:50.549 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-f[66535]: starting mds.f at 2024-03-24T03:11:50.549 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i[67156]: ignoring --setuser ceph since I am not root 2024-03-24T03:11:50.549 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i[67156]: ignoring --setgroup ceph since I am not root 2024-03-24T03:11:50.549 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mds[67160]: -- 172.21.15.188:0/406327994 <== mon.0 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c547edba40 con 0x55c547eae800 2024-03-24T03:11:50.549 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i[67156]: starting mds.i at 2024-03-24T03:11:50.549 INFO:journalctl@ceph.mds.l.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-l[66224]: ignoring --setuser ceph since I am not root 2024-03-24T03:11:50.549 INFO:journalctl@ceph.mds.l.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-l[66224]: ignoring --setgroup ceph since I am not root 2024-03-24T03:11:50.549 INFO:journalctl@ceph.mds.l.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mds[66228]: -- 172.21.15.188:0/2941216975 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562e5340da40 con 0x562e533e0c00 2024-03-24T03:11:50.550 INFO:journalctl@ceph.mds.l.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-l[66224]: starting mds.l at 2024-03-24T03:11:50.552 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b[66492]: ignoring --setuser ceph since I am not root 2024-03-24T03:11:50.552 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b[66492]: ignoring --setgroup ceph since I am not root 2024-03-24T03:11:50.552 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mds[66496]: -- 172.21.15.19:0/2284358912 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ed544cfa40 con 0x55ed544a3000 2024-03-24T03:11:50.552 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b[66492]: starting mds.b at 2024-03-24T03:11:50.553 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k[65548]: ignoring --setuser ceph since I am not root 2024-03-24T03:11:50.553 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k[65548]: ignoring --setgroup ceph since I am not root 2024-03-24T03:11:50.553 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mds[65552]: -- 172.21.15.19:0/693959342 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d39d99ba40 con 0x55d39d96ec00 2024-03-24T03:11:50.553 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k[65548]: starting mds.k at 2024-03-24T03:11:50.553 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e[65869]: ignoring --setuser ceph since I am not root 2024-03-24T03:11:50.553 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e[65869]: ignoring --setgroup ceph since I am not root 2024-03-24T03:11:50.553 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mds[65873]: -- 172.21.15.19:0/2838092585 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56388eab5a40 con 0x56388ea88c00 2024-03-24T03:11:50.553 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e[65869]: starting mds.e at 2024-03-24T03:11:50.553 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h[66184]: ignoring --setuser ceph since I am not root 2024-03-24T03:11:50.554 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h[66184]: ignoring --setgroup ceph since I am not root 2024-03-24T03:11:50.554 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mds[66188]: -- 172.21.15.19:0/2046780221 <== mon.2 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5571e5e97a40 con 0x5571e5e6a800 2024-03-24T03:11:50.554 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h[66184]: starting mds.h at 2024-03-24T03:11:50.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2500974228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:11:50.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2544453731' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-24T03:11:50.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T03:11:50.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T03:11:50.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2544453731' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-24T03:11:50.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: osdmap e104: 12 total, 12 up, 12 in 2024-03-24T03:11:50.706 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-24T03:11:50.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-24T03:11:50.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-24T03:11:50.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-24T03:11:50.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-24T03:11:50.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-24T03:11:50.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-24T03:11:50.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-24T03:11:50.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-24T03:11:50.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-24T03:11:50.707 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:50 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/230532049' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T03:11:50.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2500974228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:11:50.820 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2544453731' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2544453731' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: osdmap e104: 12 total, 12 up, 12 in 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-24T03:11:50.821 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-24T03:11:50.822 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:50 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/230532049' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T03:11:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2500974228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T03:11:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2544453731' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-24T03:11:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T03:11:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T03:11:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2544453731' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-24T03:11:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: osdmap e104: 12 total, 12 up, 12 in 2024-03-24T03:11:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-24T03:11:51.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-24T03:11:51.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-24T03:11:51.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-24T03:11:51.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-24T03:11:51.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-24T03:11:51.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-24T03:11:51.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-24T03:11:51.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-24T03:11:51.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-24T03:11:51.049 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:50 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/230532049' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T03:11:51.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.156+0000 7f668cff9640 1 -- 172.21.15.17:0/230532049 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f667809c1a0 con 0x7f6690069260 2024-03-24T03:11:51.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.160+0000 7f669564b640 1 -- 172.21.15.17:0/230532049 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f667c0a2970 msgr2=0x7f667c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:51.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.160+0000 7f669564b640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f667c0a2970 0x7f667c0a4e30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f668000bbe0 tx=0x7f668000b390 comp rx=0 tx=0).stop 2024-03-24T03:11:51.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.160+0000 7f669564b640 1 -- 172.21.15.17:0/230532049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690069260 msgr2=0x7f6690198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:51.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.160+0000 7f669564b640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690069260 0x7f6690198440 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f667800e9f0 tx=0x7f667800edf0 comp rx=0 tx=0).stop 2024-03-24T03:11:51.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.160+0000 7f669564b640 1 -- 172.21.15.17:0/230532049 shutdown_connections 2024-03-24T03:11:51.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.160+0000 7f669564b640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f667c0a2970 0x7f667c0a4e30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:51.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.160+0000 7f669564b640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66901076f0 0x7f669019c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:51.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.160+0000 7f669564b640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690069260 0x7f6690198440 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:51.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.160+0000 7f669564b640 1 --2- 172.21.15.17:0/230532049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66900688a0 0x7f6690197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:51.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.160+0000 7f669564b640 1 -- 172.21.15.17:0/230532049 >> 172.21.15.17:0/230532049 conn(0x7f66900776a0 msgr2=0x7f66900fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:51.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.160+0000 7f669564b640 1 -- 172.21.15.17:0/230532049 shutdown_connections 2024-03-24T03:11:51.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.160+0000 7f669564b640 1 -- 172.21.15.17:0/230532049 wait complete. 2024-03-24T03:11:51.173 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-24T03:11:51.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.282+0000 7f6158aff640 1 -- 172.21.15.17:0/2044461121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6154100540 msgr2=0x7f61541009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.282+0000 7f6158aff640 1 --2- 172.21.15.17:0/2044461121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6154100540 0x7f61541009c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6148009900 tx=0x7f614802bc30 comp rx=0 tx=0).stop 2024-03-24T03:11:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.282+0000 7f6158aff640 1 -- 172.21.15.17:0/2044461121 shutdown_connections 2024-03-24T03:11:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.282+0000 7f6158aff640 1 --2- 172.21.15.17:0/2044461121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6154100f00 0x7f615410d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.282+0000 7f6158aff640 1 --2- 172.21.15.17:0/2044461121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6154100540 0x7f61541009c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.282+0000 7f6158aff640 1 --2- 172.21.15.17:0/2044461121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6154106600 0x7f61541069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.282+0000 7f6158aff640 1 -- 172.21.15.17:0/2044461121 >> 172.21.15.17:0/2044461121 conn(0x7f6154077750 msgr2=0x7f6154077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.282+0000 7f6158aff640 1 -- 172.21.15.17:0/2044461121 shutdown_connections 2024-03-24T03:11:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.282+0000 7f6158aff640 1 -- 172.21.15.17:0/2044461121 wait complete. 2024-03-24T03:11:51.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f6158aff640 1 Processor -- start 2024-03-24T03:11:51.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f6158aff640 1 -- start start 2024-03-24T03:11:51.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f6158aff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6154100540 0x7f6154197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f6158aff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6154100f00 0x7f61541983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f6158aff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6154106600 0x7f615419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f6158aff640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61541108e0 con 0x7f6154100f00 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f6158aff640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6154110760 con 0x7f6154106600 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f6158aff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6154110a60 con 0x7f6154100540 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f6151d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6154100f00 0x7f61541983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f6151d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6154100f00 0x7f61541983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48168/0 (socket says 172.21.15.17:48168) 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f6151d9b640 1 -- 172.21.15.17:0/3729675020 learned_addr learned my addr 172.21.15.17:0/3729675020 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f6152d9d640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6154106600 0x7f615419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.283+0000 7f615259c640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6154100540 0x7f6154197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.284+0000 7f6151d9b640 1 -- 172.21.15.17:0/3729675020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6154100540 msgr2=0x7f6154197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.284+0000 7f6151d9b640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6154100540 0x7f6154197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.284+0000 7f6151d9b640 1 -- 172.21.15.17:0/3729675020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6154106600 msgr2=0x7f615419c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.284+0000 7f6151d9b640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6154106600 0x7f615419c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.284+0000 7f6151d9b640 1 -- 172.21.15.17:0/3729675020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615419ce50 con 0x7f6154100f00 2024-03-24T03:11:51.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.284+0000 7f615259c640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6154100540 0x7f6154197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T03:11:51.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.284+0000 7f6151d9b640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6154100f00 0x7f61541983a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6148005940 tx=0x7f61480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:51.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.284+0000 7f61437fe640 1 -- 172.21.15.17:0/3729675020 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148041070 con 0x7f6154100f00 2024-03-24T03:11:51.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.284+0000 7f61437fe640 1 -- 172.21.15.17:0/3729675020 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f614802ccc0 con 0x7f6154100f00 2024-03-24T03:11:51.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.284+0000 7f6158aff640 1 -- 172.21.15.17:0/3729675020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615419d0e0 con 0x7f6154100f00 2024-03-24T03:11:51.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.284+0000 7f61437fe640 1 -- 172.21.15.17:0/3729675020 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148003d80 con 0x7f6154100f00 2024-03-24T03:11:51.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.284+0000 7f6158aff640 1 -- 172.21.15.17:0/3729675020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61541a49d0 con 0x7f6154100f00 2024-03-24T03:11:51.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.285+0000 7f6158aff640 1 -- 172.21.15.17:0/3729675020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6154068490 con 0x7f6154100f00 2024-03-24T03:11:51.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.286+0000 7f61437fe640 1 -- 172.21.15.17:0/3729675020 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6148044070 con 0x7f6154100f00 2024-03-24T03:11:51.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.287+0000 7f61437fe640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f613c0a2970 0x7f613c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:51.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.287+0000 7f615259c640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f613c0a2970 0x7f613c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:51.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.287+0000 7f61437fe640 1 -- 172.21.15.17:0/3729675020 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 10050+0+0 (secure 0 0 0) 0x7f61480f1290 con 0x7f6154100f00 2024-03-24T03:11:51.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.287+0000 7f615259c640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f613c0a2970 0x7f613c0a4e30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f61440058c0 tx=0x7f6144005d30 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:51.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.291+0000 7f61437fe640 1 -- 172.21.15.17:0/3729675020 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f61480bdaa0 con 0x7f6154100f00 2024-03-24T03:11:51.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:51.492+0000 7f6158aff640 1 -- 172.21.15.17:0/3729675020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f61541a4bd0 con 0x7f6154100f00 2024-03-24T03:11:51.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: pgmap v336: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 11 op/s 2024-03-24T03:11:51.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T03:11:51.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T03:11:51.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: Cluster is now healthy 2024-03-24T03:11:51.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/230532049' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-24T03:11:51.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.188:6836/3159657428,v1:172.21.15.188:6838/3159657428] up:boot 2024-03-24T03:11:51.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.17:6840/4170008187,v1:172.21.15.17:6841/4170008187] up:boot 2024-03-24T03:11:51.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.19:6832/1758837377,v1:172.21.15.19:6833/1758837377] up:boot 2024-03-24T03:11:51.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.188:6837/48927483,v1:172.21.15.188:6839/48927483] up:boot 2024-03-24T03:11:51.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.17:6838/698774444,v1:172.21.15.17:6839/698774444] up:boot 2024-03-24T03:11:51.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.188:6832/1613058161,v1:172.21.15.188:6833/1613058161] up:boot 2024-03-24T03:11:51.811 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.19:6838/3631293084,v1:172.21.15.19:6839/3631293084] up:boot 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.19:6834/3951188814,v1:172.21.15.19:6835/3951188814] up:boot 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.188:6834/4109926324,v1:172.21.15.188:6835/4109926324] up:boot 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: mds.? [v2:172.21.15.19:6836/2956603364,v1:172.21.15.19:6837/2956603364] up:boot 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: fsmap 12 up:standby 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-24T03:11:51.812 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:51 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3729675020' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T03:11:51.821 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: pgmap v336: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 11 op/s 2024-03-24T03:11:51.821 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T03:11:51.821 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T03:11:51.821 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: Cluster is now healthy 2024-03-24T03:11:51.821 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/230532049' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-24T03:11:51.821 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.188:6836/3159657428,v1:172.21.15.188:6838/3159657428] up:boot 2024-03-24T03:11:51.821 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.17:6840/4170008187,v1:172.21.15.17:6841/4170008187] up:boot 2024-03-24T03:11:51.821 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.19:6832/1758837377,v1:172.21.15.19:6833/1758837377] up:boot 2024-03-24T03:11:51.821 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.188:6837/48927483,v1:172.21.15.188:6839/48927483] up:boot 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.17:6838/698774444,v1:172.21.15.17:6839/698774444] up:boot 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.188:6832/1613058161,v1:172.21.15.188:6833/1613058161] up:boot 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.19:6838/3631293084,v1:172.21.15.19:6839/3631293084] up:boot 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.19:6834/3951188814,v1:172.21.15.19:6835/3951188814] up:boot 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.188:6834/4109926324,v1:172.21.15.188:6835/4109926324] up:boot 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: mds.? [v2:172.21.15.19:6836/2956603364,v1:172.21.15.19:6837/2956603364] up:boot 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: fsmap 12 up:standby 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-24T03:11:51.822 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-24T03:11:51.823 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-24T03:11:51.823 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-24T03:11:51.823 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-24T03:11:51.823 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-24T03:11:51.823 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:51 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3729675020' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T03:11:51.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: pgmap v336: 129 pgs: 129 active+clean; 12 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 11 op/s 2024-03-24T03:11:51.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T03:11:51.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: Cluster is now healthy 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/230532049' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.188:6836/3159657428,v1:172.21.15.188:6838/3159657428] up:boot 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.17:6840/4170008187,v1:172.21.15.17:6841/4170008187] up:boot 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.19:6832/1758837377,v1:172.21.15.19:6833/1758837377] up:boot 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.188:6837/48927483,v1:172.21.15.188:6839/48927483] up:boot 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.17:6838/698774444,v1:172.21.15.17:6839/698774444] up:boot 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.188:6832/1613058161,v1:172.21.15.188:6833/1613058161] up:boot 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.19:6838/3631293084,v1:172.21.15.19:6839/3631293084] up:boot 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.19:6834/3951188814,v1:172.21.15.19:6835/3951188814] up:boot 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.188:6834/4109926324,v1:172.21.15.188:6835/4109926324] up:boot 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: mds.? [v2:172.21.15.19:6836/2956603364,v1:172.21.15.19:6837/2956603364] up:boot 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: fsmap 12 up:standby 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-24T03:11:51.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-24T03:11:51.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-24T03:11:51.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-24T03:11:51.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-24T03:11:51.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-24T03:11:51.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-24T03:11:51.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-24T03:11:51.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-24T03:11:51.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: from='mgr.25334 172.21.15.17:0/2977109728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-24T03:11:51.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:51 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3729675020' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T03:11:52.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.158+0000 7f61437fe640 1 -- 172.21.15.17:0/3729675020 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 181+0+0 (secure 0 0 0) 0x7f61480bd3a0 con 0x7f6154100f00 2024-03-24T03:11:52.158 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-24T03:11:52.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.162+0000 7f6158aff640 1 -- 172.21.15.17:0/3729675020 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f613c0a2970 msgr2=0x7f613c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.162+0000 7f6158aff640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f613c0a2970 0x7f613c0a4e30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f61440058c0 tx=0x7f6144005d30 comp rx=0 tx=0).stop 2024-03-24T03:11:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.162+0000 7f6158aff640 1 -- 172.21.15.17:0/3729675020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6154100f00 msgr2=0x7f61541983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.162+0000 7f6158aff640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6154100f00 0x7f61541983a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6148005940 tx=0x7f61480042f0 comp rx=0 tx=0).stop 2024-03-24T03:11:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.162+0000 7f6158aff640 1 -- 172.21.15.17:0/3729675020 shutdown_connections 2024-03-24T03:11:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.162+0000 7f6158aff640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f613c0a2970 0x7f613c0a4e30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.162+0000 7f6158aff640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6154106600 0x7f615419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.162+0000 7f6158aff640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6154100f00 0x7f61541983a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.162+0000 7f6158aff640 1 --2- 172.21.15.17:0/3729675020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6154100540 0x7f6154197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.162+0000 7f6158aff640 1 -- 172.21.15.17:0/3729675020 >> 172.21.15.17:0/3729675020 conn(0x7f6154077750 msgr2=0x7f61540ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.162+0000 7f6158aff640 1 -- 172.21.15.17:0/3729675020 shutdown_connections 2024-03-24T03:11:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.162+0000 7f6158aff640 1 -- 172.21.15.17:0/3729675020 wait complete. 2024-03-24T03:11:52.176 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-24T03:11:52.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.291+0000 7f335dd1b640 1 -- 172.21.15.17:0/766128317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3358100f80 msgr2=0x7f335810d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.291+0000 7f335dd1b640 1 --2- 172.21.15.17:0/766128317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3358100f80 0x7f335810d490 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3350009cd0 tx=0x7f335002c0f0 comp rx=0 tx=0).stop 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.291+0000 7f335dd1b640 1 -- 172.21.15.17:0/766128317 shutdown_connections 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.291+0000 7f335dd1b640 1 --2- 172.21.15.17:0/766128317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3358100f80 0x7f335810d490 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.291+0000 7f335dd1b640 1 --2- 172.21.15.17:0/766128317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33581005c0 0x7f3358100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.291+0000 7f335dd1b640 1 --2- 172.21.15.17:0/766128317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358106660 0x7f3358106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.291+0000 7f335dd1b640 1 -- 172.21.15.17:0/766128317 >> 172.21.15.17:0/766128317 conn(0x7f3358077700 msgr2=0x7f3358077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.291+0000 7f335dd1b640 1 -- 172.21.15.17:0/766128317 shutdown_connections 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.291+0000 7f335dd1b640 1 -- 172.21.15.17:0/766128317 wait complete. 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f335dd1b640 1 Processor -- start 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f335dd1b640 1 -- start start 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f335dd1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33581005c0 0x7f3358197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f335dd1b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3358100f80 0x7f3358198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f335dd1b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3358106660 0x7f335819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f335dd1b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33581109e0 con 0x7f3358100f80 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f335dd1b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3358110860 con 0x7f33581005c0 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f335dd1b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3358110b60 con 0x7f3358106660 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3358100f80 0x7f3358198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f33577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33581005c0 0x7f3358197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f3357fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3358106660 0x7f335819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f33577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33581005c0 0x7f3358197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.17:35584/0 (socket says 172.21.15.17:35584) 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.292+0000 7f33577fe640 1 -- 172.21.15.17:0/2118824125 learned_addr learned my addr 172.21.15.17:0/2118824125 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f3356ffd640 1 -- 172.21.15.17:0/2118824125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3358106660 msgr2=0x7f335819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f3356ffd640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3358106660 0x7f335819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f3356ffd640 1 -- 172.21.15.17:0/2118824125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33581005c0 msgr2=0x7f3358197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f3356ffd640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33581005c0 0x7f3358197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f3356ffd640 1 -- 172.21.15.17:0/2118824125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335819cec0 con 0x7f3358100f80 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f33577fe640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33581005c0 0x7f3358197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:11:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f3357fff640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3358106660 0x7f335819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T03:11:52.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f3356ffd640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3358100f80 0x7f3358198410 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f334c00e9f0 tx=0x7f334c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:52.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f3354ff9640 1 -- 172.21.15.17:0/2118824125 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c00ccc0 con 0x7f3358100f80 2024-03-24T03:11:52.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f335dd1b640 1 -- 172.21.15.17:0/2118824125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335819d1b0 con 0x7f3358100f80 2024-03-24T03:11:52.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f3354ff9640 1 -- 172.21.15.17:0/2118824125 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f334c005020 con 0x7f3358100f80 2024-03-24T03:11:52.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f3354ff9640 1 -- 172.21.15.17:0/2118824125 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c0049e0 con 0x7f3358100f80 2024-03-24T03:11:52.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.293+0000 7f335dd1b640 1 -- 172.21.15.17:0/2118824125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33581a4a40 con 0x7f3358100f80 2024-03-24T03:11:52.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.294+0000 7f335dd1b640 1 -- 172.21.15.17:0/2118824125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33580688e0 con 0x7f3358100f80 2024-03-24T03:11:52.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.295+0000 7f3354ff9640 1 -- 172.21.15.17:0/2118824125 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f334c005560 con 0x7f3358100f80 2024-03-24T03:11:52.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.296+0000 7f3354ff9640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f33400a2970 0x7f33400a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:52.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.296+0000 7f33577fe640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f33400a2970 0x7f33400a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:52.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.296+0000 7f3354ff9640 1 -- 172.21.15.17:0/2118824125 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9430+0+0 (secure 0 0 0) 0x7f334c0cf1a0 con 0x7f3358100f80 2024-03-24T03:11:52.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.296+0000 7f33577fe640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f33400a2970 0x7f33400a4e30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f33480104e0 tx=0x7f334800b410 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:52.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.300+0000 7f3354ff9640 1 -- 172.21.15.17:0/2118824125 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f334c0051d0 con 0x7f3358100f80 2024-03-24T03:11:52.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:52.488+0000 7f335dd1b640 1 -- 172.21.15.17:0/2118824125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f33581a4c10 con 0x7f3358100f80 2024-03-24T03:11:53.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.164+0000 7f3354ff9640 1 -- 172.21.15.17:0/2118824125 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 169+0+0 (secure 0 0 0) 0x7f334c09bb70 con 0x7f3358100f80 2024-03-24T03:11:53.163 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-24T03:11:53.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.168+0000 7f335dd1b640 1 -- 172.21.15.17:0/2118824125 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f33400a2970 msgr2=0x7f33400a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:53.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.168+0000 7f335dd1b640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f33400a2970 0x7f33400a4e30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f33480104e0 tx=0x7f334800b410 comp rx=0 tx=0).stop 2024-03-24T03:11:53.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.168+0000 7f335dd1b640 1 -- 172.21.15.17:0/2118824125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3358100f80 msgr2=0x7f3358198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:53.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.168+0000 7f335dd1b640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3358100f80 0x7f3358198410 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f334c00e9f0 tx=0x7f334c00eeb0 comp rx=0 tx=0).stop 2024-03-24T03:11:53.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.168+0000 7f335dd1b640 1 -- 172.21.15.17:0/2118824125 shutdown_connections 2024-03-24T03:11:53.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.168+0000 7f335dd1b640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7f33400a2970 0x7f33400a4e30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:53.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.168+0000 7f335dd1b640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3358106660 0x7f335819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:53.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.168+0000 7f335dd1b640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3358100f80 0x7f3358198410 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:53.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.168+0000 7f335dd1b640 1 --2- 172.21.15.17:0/2118824125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33581005c0 0x7f3358197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:53.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.168+0000 7f335dd1b640 1 -- 172.21.15.17:0/2118824125 >> 172.21.15.17:0/2118824125 conn(0x7f3358077700 msgr2=0x7f33580ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:53.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.168+0000 7f335dd1b640 1 -- 172.21.15.17:0/2118824125 shutdown_connections 2024-03-24T03:11:53.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.168+0000 7f335dd1b640 1 -- 172.21.15.17:0/2118824125 wait complete. 2024-03-24T03:11:53.184 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-24T03:11:53.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.301+0000 7fb9910dc640 1 -- 172.21.15.17:0/3971100873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb98c107650 msgr2=0x7fb98c107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.301+0000 7fb9910dc640 1 --2- 172.21.15.17:0/3971100873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb98c107650 0x7fb98c107a30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009930 tx=0x7fb97c02bc30 comp rx=0 tx=0).stop 2024-03-24T03:11:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.301+0000 7fb9910dc640 1 -- 172.21.15.17:0/3971100873 shutdown_connections 2024-03-24T03:11:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.301+0000 7fb9910dc640 1 --2- 172.21.15.17:0/3971100873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c1004f0 0x7fb98c10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.301+0000 7fb9910dc640 1 --2- 172.21.15.17:0/3971100873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb98c0ffb30 0x7fb98c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.301+0000 7fb9910dc640 1 --2- 172.21.15.17:0/3971100873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb98c107650 0x7fb98c107a30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.301+0000 7fb9910dc640 1 -- 172.21.15.17:0/3971100873 >> 172.21.15.17:0/3971100873 conn(0x7fb98c076ab0 msgr2=0x7fb98c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.301+0000 7fb9910dc640 1 -- 172.21.15.17:0/3971100873 shutdown_connections 2024-03-24T03:11:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.301+0000 7fb9910dc640 1 -- 172.21.15.17:0/3971100873 wait complete. 2024-03-24T03:11:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb9910dc640 1 Processor -- start 2024-03-24T03:11:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb9910dc640 1 -- start start 2024-03-24T03:11:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb9910dc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb98c0ffb30 0x7fb98c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb9910dc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb98c1004f0 0x7fb98c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb9910dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c107650 0x7fb98c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb9910dc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c111a20 con 0x7fb98c0ffb30 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb9910dc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c1118a0 con 0x7fb98c107650 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb9910dc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c111ba0 con 0x7fb98c1004f0 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb98ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb98c0ffb30 0x7fb98c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb98ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb98c0ffb30 0x7fb98c197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48206/0 (socket says 172.21.15.17:48206) 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb98ad9d640 1 -- 172.21.15.17:0/2316084291 learned_addr learned my addr 172.21.15.17:0/2316084291 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb983fff640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb98c1004f0 0x7fb98c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.302+0000 7fb98b59e640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c107650 0x7fb98c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.303+0000 7fb98ad9d640 1 -- 172.21.15.17:0/2316084291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb98c1004f0 msgr2=0x7fb98c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.303+0000 7fb98ad9d640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb98c1004f0 0x7fb98c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.303+0000 7fb98ad9d640 1 -- 172.21.15.17:0/2316084291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c107650 msgr2=0x7fb98c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.303+0000 7fb98ad9d640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c107650 0x7fb98c19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.303+0000 7fb98ad9d640 1 -- 172.21.15.17:0/2316084291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c19cf90 con 0x7fb98c0ffb30 2024-03-24T03:11:53.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.303+0000 7fb98ad9d640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb98c0ffb30 0x7fb98c197fa0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009230 tx=0x7fb97c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:53.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.303+0000 7fb988d99640 1 -- 172.21.15.17:0/2316084291 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c041070 con 0x7fb98c0ffb30 2024-03-24T03:11:53.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.303+0000 7fb988d99640 1 -- 172.21.15.17:0/2316084291 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb97c004450 con 0x7fb98c0ffb30 2024-03-24T03:11:53.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.303+0000 7fb9910dc640 1 -- 172.21.15.17:0/2316084291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98c19d220 con 0x7fb98c0ffb30 2024-03-24T03:11:53.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.303+0000 7fb988d99640 1 -- 172.21.15.17:0/2316084291 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c034440 con 0x7fb98c0ffb30 2024-03-24T03:11:53.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.303+0000 7fb9910dc640 1 -- 172.21.15.17:0/2316084291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98c1a4b10 con 0x7fb98c0ffb30 2024-03-24T03:11:53.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.304+0000 7fb9910dc640 1 -- 172.21.15.17:0/2316084291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98c0688e0 con 0x7fb98c0ffb30 2024-03-24T03:11:53.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.305+0000 7fb988d99640 1 -- 172.21.15.17:0/2316084291 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb97c044070 con 0x7fb98c0ffb30 2024-03-24T03:11:53.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.306+0000 7fb988d99640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb9780a29f0 0x7fb9780a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T03:11:53.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.306+0000 7fb983fff640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb9780a29f0 0x7fb9780a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T03:11:53.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.306+0000 7fb988d99640 1 -- 172.21.15.17:0/2316084291 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8797+0+0 (secure 0 0 0) 0x7fb97c0f0880 con 0x7fb98c0ffb30 2024-03-24T03:11:53.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.306+0000 7fb983fff640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb9780a29f0 0x7fb9780a4eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0ff1a0 tx=0x7fb97400b410 comp rx=0 tx=0).ready entity=mgr.25334 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T03:11:53.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.311+0000 7fb988d99640 1 -- 172.21.15.17:0/2316084291 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb97c0bcf90 con 0x7fb98c0ffb30 2024-03-24T03:11:53.455 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:53 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/3729675020' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T03:11:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:53 smithi017 ceph-mon[31926]: osdmap e105: 12 total, 12 up, 12 in 2024-03-24T03:11:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:53 smithi017 ceph-mon[31926]: pgmap v338: 97 pgs: 97 active+clean; 577 KiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T03:11:53.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:53 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2118824125' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T03:11:53.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:53.494+0000 7fb9910dc640 1 -- 172.21.15.17:0/2316084291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb98c109bb0 con 0x7fb98c0ffb30 2024-03-24T03:11:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:53 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/3729675020' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T03:11:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:53 smithi188 ceph-mon[36206]: osdmap e105: 12 total, 12 up, 12 in 2024-03-24T03:11:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:53 smithi188 ceph-mon[36206]: pgmap v338: 97 pgs: 97 active+clean; 577 KiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T03:11:53.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:53 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2118824125' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T03:11:53.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:53 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/3729675020' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T03:11:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:53 smithi019 ceph-mon[35944]: osdmap e105: 12 total, 12 up, 12 in 2024-03-24T03:11:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:53 smithi019 ceph-mon[35944]: pgmap v338: 97 pgs: 97 active+clean; 577 KiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-24T03:11:53.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:53 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2118824125' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T03:11:54.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.165+0000 7fb988d99640 1 -- 172.21.15.17:0/2316084291 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v107) v1 ==== 178+0+0 (secure 0 0 0) 0x7fb97c0c1030 con 0x7fb98c0ffb30 2024-03-24T03:11:54.165 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data_ec' removed 2024-03-24T03:11:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.169+0000 7fb9910dc640 1 -- 172.21.15.17:0/2316084291 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb9780a29f0 msgr2=0x7fb9780a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.169+0000 7fb9910dc640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb9780a29f0 0x7fb9780a4eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0ff1a0 tx=0x7fb97400b410 comp rx=0 tx=0).stop 2024-03-24T03:11:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.169+0000 7fb9910dc640 1 -- 172.21.15.17:0/2316084291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb98c0ffb30 msgr2=0x7fb98c197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T03:11:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.169+0000 7fb9910dc640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb98c0ffb30 0x7fb98c197fa0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009230 tx=0x7fb97c02cc70 comp rx=0 tx=0).stop 2024-03-24T03:11:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.169+0000 7fb9910dc640 1 -- 172.21.15.17:0/2316084291 shutdown_connections 2024-03-24T03:11:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.169+0000 7fb9910dc640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.17:6800/1006945247,v1:172.21.15.17:6801/1006945247] conn(0x7fb9780a29f0 0x7fb9780a4eb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.169+0000 7fb9910dc640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c107650 0x7fb98c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.169+0000 7fb9910dc640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb98c1004f0 0x7fb98c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.169+0000 7fb9910dc640 1 --2- 172.21.15.17:0/2316084291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb98c0ffb30 0x7fb98c197fa0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T03:11:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.169+0000 7fb9910dc640 1 -- 172.21.15.17:0/2316084291 >> 172.21.15.17:0/2316084291 conn(0x7fb98c076ab0 msgr2=0x7fb98c104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T03:11:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.169+0000 7fb9910dc640 1 -- 172.21.15.17:0/2316084291 shutdown_connections 2024-03-24T03:11:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-24T03:11:54.169+0000 7fb9910dc640 1 -- 172.21.15.17:0/2316084291 wait complete. 2024-03-24T03:11:54.189 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T03:11:54.219 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T03:11:54.257 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T03:11:54.291 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-24T03:11:54.291 DEBUG:teuthology.orchestra.run.smithi017:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-24T03:11:54.306 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-24T03:11:54.322 DEBUG:teuthology.orchestra.run.smithi188:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-24T03:11:54.346 INFO:tasks.cephadm:Stopping all daemons... 2024-03-24T03:11:54.346 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-24T03:11:54.346 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.a 2024-03-24T03:11:54.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:54 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2118824125' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T03:11:54.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:54 smithi017 ceph-mon[31926]: osdmap e106: 12 total, 12 up, 12 in 2024-03-24T03:11:54.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:54 smithi017 ceph-mon[31926]: from='client.? 172.21.15.17:0/2316084291' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T03:11:54.456 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:54 smithi017 systemd[1]: Stopping Ceph mon.a for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:11:54.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:54 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2118824125' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T03:11:54.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:54 smithi188 ceph-mon[36206]: osdmap e106: 12 total, 12 up, 12 in 2024-03-24T03:11:54.548 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:54 smithi188 ceph-mon[36206]: from='client.? 172.21.15.17:0/2316084291' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T03:11:54.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:54 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2118824125' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T03:11:54.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:54 smithi019 ceph-mon[35944]: osdmap e106: 12 total, 12 up, 12 in 2024-03-24T03:11:54.570 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:54 smithi019 ceph-mon[35944]: from='client.? 172.21.15.17:0/2316084291' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T03:11:54.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:54 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a[31922]: 2024-03-24T03:11:54.656+0000 7f77dd660700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:11:54.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:54 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a[31922]: 2024-03-24T03:11:54.656+0000 7f77dd660700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-24T03:11:54.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:54 smithi017 podman[92163]: 2024-03-24 03:11:54.770197098 +0000 UTC m=+0.227407886 container died 2644ce8a27f2ce658380e51b2d1b4f55cf03fbc030ad0c47312c0abc4ad41b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-24T03:11:55.408 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:55 smithi017 podman[92163]: 2024-03-24 03:11:55.182797375 +0000 UTC m=+0.640008171 container cleanup 2644ce8a27f2ce658380e51b2d1b4f55cf03fbc030ad0c47312c0abc4ad41b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-24T03:11:55.408 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:55 smithi017 bash[92163]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a 2024-03-24T03:11:55.685 INFO:journalctl@ceph.mon.a.smithi017.stdout:Mar 24 03:11:55 smithi017 podman[92177]: 2024-03-24 03:11:55.409388164 +0000 UTC m=+0.638739115 container remove 2644ce8a27f2ce658380e51b2d1b4f55cf03fbc030ad0c47312c0abc4ad41b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-a, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T03:11:55.710 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.a.service' 2024-03-24T03:11:55.760 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:11:55.760 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-24T03:11:55.760 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-24T03:11:55.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.b 2024-03-24T03:11:56.070 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:55 smithi019 systemd[1]: Stopping Ceph mon.b for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:11:56.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:56 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-b[35940]: 2024-03-24T03:11:56.092+0000 7f397fb41700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:11:56.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:56 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-b[35940]: 2024-03-24T03:11:56.092+0000 7f397fb41700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-24T03:11:56.571 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:56 smithi019 podman[71913]: 2024-03-24 03:11:56.30773911 +0000 UTC m=+0.337233384 container died 2b65ede2e6d9b715a07dbe1de8e6fcca4e1cd0bf59e42fde360d159376cec319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2) 2024-03-24T03:11:56.960 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:56 smithi019 podman[71913]: 2024-03-24 03:11:56.684631123 +0000 UTC m=+0.714125375 container cleanup 2b65ede2e6d9b715a07dbe1de8e6fcca4e1cd0bf59e42fde360d159376cec319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-24T03:11:56.960 INFO:journalctl@ceph.mon.b.smithi019.stdout:Mar 24 03:11:56 smithi019 bash[71913]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-b 2024-03-24T03:11:57.073 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.b.service' 2024-03-24T03:11:57.123 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:11:57.123 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-24T03:11:57.124 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-24T03:11:57.124 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.c 2024-03-24T03:11:57.476 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:57 smithi188 systemd[1]: Stopping Ceph mon.c for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:11:57.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:57 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-c[36202]: 2024-03-24T03:11:57.476+0000 7ff0b25c1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:11:57.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:57 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-c[36202]: 2024-03-24T03:11:57.476+0000 7ff0b25c1700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-24T03:11:57.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:57 smithi188 podman[72507]: 2024-03-24 03:11:57.609715509 +0000 UTC m=+0.287770586 container died 6e9e3f6184421e1100758624dfcd714c15c89e0b98fe408b2a47b830922bf2d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-c, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-24T03:11:58.280 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:58 smithi188 podman[72507]: 2024-03-24 03:11:58.004324305 +0000 UTC m=+0.682379367 container cleanup 6e9e3f6184421e1100758624dfcd714c15c89e0b98fe408b2a47b830922bf2d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T03:11:58.280 INFO:journalctl@ceph.mon.c.smithi188.stdout:Mar 24 03:11:58 smithi188 bash[72507]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mon-c 2024-03-24T03:11:58.416 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mon.c.service' 2024-03-24T03:11:58.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:11:58.475 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-24T03:11:58.475 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-24T03:11:58.475 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.x 2024-03-24T03:11:58.788 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:11:58 smithi017 systemd[1]: Stopping Ceph mgr.x for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:11:59.121 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:11:58 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:11:58.788+0000 7f007060f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:11:59.121 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:11:58 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x[32161]: 2024-03-24T03:11:58.788+0000 7f007060f700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-24T03:11:59.121 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:11:58 smithi017 podman[92284]: 2024-03-24 03:11:58.819024051 +0000 UTC m=+0.140376977 container died 13a48e6ed072e3aca460a802d9718e5995dd96f48915160717126c56405e342b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-24T03:11:59.456 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:11:59 smithi017 podman[92284]: 2024-03-24 03:11:59.213635232 +0000 UTC m=+0.534988155 container cleanup 13a48e6ed072e3aca460a802d9718e5995dd96f48915160717126c56405e342b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-24T03:11:59.456 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Mar 24 03:11:59 smithi017 bash[92284]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-x 2024-03-24T03:11:59.728 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-24T03:11:59.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:11:59.765 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-24T03:11:59.765 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-24T03:11:59.765 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.y 2024-03-24T03:12:00.070 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:11:59 smithi019 systemd[1]: Stopping Ceph mgr.y for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:12:00.381 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:12:00 smithi019 podman[72034]: 2024-03-24 03:12:00.101687506 +0000 UTC m=+0.134968390 container died 283658810b5c98dc8ee84f381c73e1d759f72e4e9c45ad5338662386d1aa557d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T03:12:00.757 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:12:00 smithi019 podman[72034]: 2024-03-24 03:12:00.465072666 +0000 UTC m=+0.498353552 container cleanup 283658810b5c98dc8ee84f381c73e1d759f72e4e9c45ad5338662386d1aa557d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-24T03:12:00.757 INFO:journalctl@ceph.mgr.y.smithi019.stdout:Mar 24 03:12:00 smithi019 bash[72034]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-y 2024-03-24T03:12:00.887 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-24T03:12:00.928 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:12:00.928 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-24T03:12:00.928 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-24T03:12:00.928 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.z 2024-03-24T03:12:01.279 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:12:00 smithi188 systemd[1]: Stopping Ceph mgr.z for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:12:01.548 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:12:01 smithi188 podman[72630]: 2024-03-24 03:12:01.279594119 +0000 UTC m=+0.157306128 container died b80ef274913f6c0f34974588198694a5c62e9378fcf2d10885617d93e5d60dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:12:01.911 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:12:01 smithi188 podman[72630]: 2024-03-24 03:12:01.677312214 +0000 UTC m=+0.555024222 container cleanup b80ef274913f6c0f34974588198694a5c62e9378fcf2d10885617d93e5d60dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-03-24T03:12:01.911 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Mar 24 03:12:01 smithi188 bash[72630]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mgr-z 2024-03-24T03:12:02.044 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mgr.z.service' 2024-03-24T03:12:02.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:12:02.081 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-24T03:12:02.082 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-24T03:12:02.082 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.0 2024-03-24T03:12:02.401 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:02 smithi017 systemd[1]: Stopping Ceph osd.0 for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:12:02.706 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:02 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0[46572]: 2024-03-24T03:12:02.402+0000 7f8f2667d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:12:02.706 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:02 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0[46572]: 2024-03-24T03:12:02.402+0000 7f8f2667d700 -1 osd.0 107 *** Got signal Terminated *** 2024-03-24T03:12:02.706 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:02 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0[46572]: 2024-03-24T03:12:02.402+0000 7f8f2667d700 -1 osd.0 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T03:12:08.206 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:07 smithi017 podman[92406]: 2024-03-24 03:12:07.877277825 +0000 UTC m=+5.594556742 container died 823ceb74ebc12fd27bdf1ccde91f58b1a0bbcb73737cff41658cd6afb3690a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-24T03:12:08.620 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:08 smithi017 podman[92406]: 2024-03-24 03:12:08.394630842 +0000 UTC m=+6.111909770 container cleanup 823ceb74ebc12fd27bdf1ccde91f58b1a0bbcb73737cff41658cd6afb3690a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-24T03:12:08.620 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:08 smithi017 bash[92406]: ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0 2024-03-24T03:12:08.956 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:08 smithi017 podman[92424]: 2024-03-24 03:12:08.621152083 +0000 UTC m=+0.743612970 container remove 823ceb74ebc12fd27bdf1ccde91f58b1a0bbcb73737cff41658cd6afb3690a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-24T03:12:09.207 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:09 smithi017 podman[92517]: 2024-03-24 03:12:08.973776345 +0000 UTC m=+0.018609813 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T03:12:09.207 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:09 smithi017 podman[92517]: 2024-03-24 03:12:09.08881045 +0000 UTC m=+0.133643975 container create 5219cad2a0b8528091a2fe0bdd63f179603dce43d42ef8c0973a2253d019f788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T03:12:09.706 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:09 smithi017 podman[92517]: 2024-03-24 03:12:09.425134874 +0000 UTC m=+0.469968342 container init 5219cad2a0b8528091a2fe0bdd63f179603dce43d42ef8c0973a2253d019f788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T03:12:09.706 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:09 smithi017 podman[92517]: 2024-03-24 03:12:09.42825689 +0000 UTC m=+0.473090340 container start 5219cad2a0b8528091a2fe0bdd63f179603dce43d42ef8c0973a2253d019f788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:12:09.706 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:09 smithi017 podman[92517]: 2024-03-24 03:12:09.525080111 +0000 UTC m=+0.569913568 container attach 5219cad2a0b8528091a2fe0bdd63f179603dce43d42ef8c0973a2253d019f788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-24T03:12:10.086 INFO:journalctl@ceph.osd.0.smithi017.stdout:Mar 24 03:12:09 smithi017 podman[92517]: 2024-03-24 03:12:09.733508731 +0000 UTC m=+0.778342188 container died 5219cad2a0b8528091a2fe0bdd63f179603dce43d42ef8c0973a2253d019f788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-24T03:12:10.487 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.0.service' 2024-03-24T03:12:10.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:12:10.534 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-24T03:12:10.535 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-24T03:12:10.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.1 2024-03-24T03:12:10.820 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:10 smithi019 systemd[1]: Stopping Ceph osd.1 for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:12:11.320 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:10 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1[40541]: 2024-03-24T03:12:10.868+0000 7f13c6e77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:12:11.320 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:10 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1[40541]: 2024-03-24T03:12:10.868+0000 7f13c6e77700 -1 osd.1 107 *** Got signal Terminated *** 2024-03-24T03:12:11.321 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:10 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1[40541]: 2024-03-24T03:12:10.868+0000 7f13c6e77700 -1 osd.1 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T03:12:16.622 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:16 smithi019 podman[72159]: 2024-03-24 03:12:16.349387878 +0000 UTC m=+5.604412357 container died 9dd6c1770b03b790f9142334bbff7bb4da017317b694f9feba9b4db2a23429ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-24T03:12:17.029 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:16 smithi019 podman[72159]: 2024-03-24 03:12:16.703430431 +0000 UTC m=+5.958454909 container cleanup 9dd6c1770b03b790f9142334bbff7bb4da017317b694f9feba9b4db2a23429ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-24T03:12:17.029 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:16 smithi019 bash[72159]: ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1 2024-03-24T03:12:17.320 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:17 smithi019 podman[72177]: 2024-03-24 03:12:17.029896514 +0000 UTC m=+0.673706793 container remove 9dd6c1770b03b790f9142334bbff7bb4da017317b694f9feba9b4db2a23429ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-24T03:12:17.674 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:17 smithi019 podman[72272]: 2024-03-24 03:12:17.29125446 +0000 UTC m=+0.018489815 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T03:12:17.675 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:17 smithi019 podman[72272]: 2024-03-24 03:12:17.405863948 +0000 UTC m=+0.133099284 container create 71fbae53ad56f75044d29df218e21e9d9945d995b644dbaba3ed53ebf4bc9b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T03:12:17.979 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:17 smithi019 podman[72272]: 2024-03-24 03:12:17.675412166 +0000 UTC m=+0.402647516 container init 71fbae53ad56f75044d29df218e21e9d9945d995b644dbaba3ed53ebf4bc9b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T03:12:17.979 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:17 smithi019 podman[72272]: 2024-03-24 03:12:17.67967165 +0000 UTC m=+0.406906993 container start 71fbae53ad56f75044d29df218e21e9d9945d995b644dbaba3ed53ebf4bc9b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2) 2024-03-24T03:12:17.979 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:17 smithi019 podman[72272]: 2024-03-24 03:12:17.742211065 +0000 UTC m=+0.469446408 container attach 71fbae53ad56f75044d29df218e21e9d9945d995b644dbaba3ed53ebf4bc9b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-03-24T03:12:18.298 INFO:journalctl@ceph.osd.1.smithi019.stdout:Mar 24 03:12:17 smithi019 podman[72272]: 2024-03-24 03:12:17.97996673 +0000 UTC m=+0.707202074 container died 71fbae53ad56f75044d29df218e21e9d9945d995b644dbaba3ed53ebf4bc9b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-24T03:12:18.630 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.1.service' 2024-03-24T03:12:18.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:12:18.679 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-24T03:12:18.680 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-24T03:12:18.680 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.2 2024-03-24T03:12:18.996 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:18 smithi188 systemd[1]: Stopping Ceph osd.2 for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:12:19.298 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:18 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2[41347]: 2024-03-24T03:12:18.995+0000 7fdddd339700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:12:19.298 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:18 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2[41347]: 2024-03-24T03:12:18.995+0000 7fdddd339700 -1 osd.2 107 *** Got signal Terminated *** 2024-03-24T03:12:19.298 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:18 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2[41347]: 2024-03-24T03:12:18.995+0000 7fdddd339700 -1 osd.2 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T03:12:24.637 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:24 smithi188 podman[72753]: 2024-03-24 03:12:24.33479227 +0000 UTC m=+5.463525647 container died f106279c622853913d6192330f9e70eae1a17f8fcecb52b779639d0c6589e822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:12:24.955 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:24 smithi188 podman[72753]: 2024-03-24 03:12:24.729318581 +0000 UTC m=+5.858051946 container cleanup f106279c622853913d6192330f9e70eae1a17f8fcecb52b779639d0c6589e822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T03:12:24.955 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:24 smithi188 bash[72753]: ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2 2024-03-24T03:12:25.259 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:24 smithi188 podman[72770]: 2024-03-24 03:12:24.955999931 +0000 UTC m=+0.624943558 container remove f106279c622853913d6192330f9e70eae1a17f8fcecb52b779639d0c6589e822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T03:12:25.517 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:25 smithi188 podman[72863]: 2024-03-24 03:12:25.163177474 +0000 UTC m=+0.017931124 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T03:12:25.517 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:25 smithi188 podman[72863]: 2024-03-24 03:12:25.265216837 +0000 UTC m=+0.119970465 container create be9ec403f7a1bbb3733ba13902ed598f87a1bd449d05d574aca60bcf165a6694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:12:25.517 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:25 smithi188 podman[72863]: 2024-03-24 03:12:25.517900215 +0000 UTC m=+0.372653851 container init be9ec403f7a1bbb3733ba13902ed598f87a1bd449d05d574aca60bcf165a6694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-24T03:12:25.798 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:25 smithi188 podman[72863]: 2024-03-24 03:12:25.521412841 +0000 UTC m=+0.376166469 container start be9ec403f7a1bbb3733ba13902ed598f87a1bd449d05d574aca60bcf165a6694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-24T03:12:25.798 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:25 smithi188 podman[72863]: 2024-03-24 03:12:25.618059815 +0000 UTC m=+0.472813467 container attach be9ec403f7a1bbb3733ba13902ed598f87a1bd449d05d574aca60bcf165a6694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:12:26.120 INFO:journalctl@ceph.osd.2.smithi188.stdout:Mar 24 03:12:25 smithi188 podman[72863]: 2024-03-24 03:12:25.807602032 +0000 UTC m=+0.662355686 container died be9ec403f7a1bbb3733ba13902ed598f87a1bd449d05d574aca60bcf165a6694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:12:26.470 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.2.service' 2024-03-24T03:12:26.514 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:12:26.515 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-24T03:12:26.515 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-24T03:12:26.515 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.3 2024-03-24T03:12:26.823 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:26 smithi017 systemd[1]: Stopping Ceph osd.3 for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:12:27.206 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:26 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3[53083]: 2024-03-24T03:12:26.823+0000 7faffe63f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:12:27.206 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:26 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3[53083]: 2024-03-24T03:12:26.823+0000 7faffe63f700 -1 osd.3 107 *** Got signal Terminated *** 2024-03-24T03:12:27.206 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:26 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3[53083]: 2024-03-24T03:12:26.823+0000 7faffe63f700 -1 osd.3 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T03:12:29.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:28 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:28.782+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:30.047 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:29 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:29.749+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:31.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:30 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:30.792+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:32.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:31 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:31.807+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:32.524 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:32 smithi017 podman[92640]: 2024-03-24 03:12:32.210248468 +0000 UTC m=+5.498636956 container died 7a991e11c0463126d80d7677b2b01770f71f0f8f94d7e67fa90b32082875f0d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-24T03:12:32.879 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:32 smithi017 podman[92640]: 2024-03-24 03:12:32.616026046 +0000 UTC m=+5.904414544 container cleanup 7a991e11c0463126d80d7677b2b01770f71f0f8f94d7e67fa90b32082875f0d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-03-24T03:12:32.880 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:32 smithi017 bash[92640]: ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3 2024-03-24T03:12:33.101 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:32 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:32.805+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:33.194 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:32 smithi017 podman[92657]: 2024-03-24 03:12:32.880662389 +0000 UTC m=+0.661299777 container remove 7a991e11c0463126d80d7677b2b01770f71f0f8f94d7e67fa90b32082875f0d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-24T03:12:33.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:32 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:32.955+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:33.445 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:33 smithi017 podman[92752]: 2024-03-24 03:12:33.099510856 +0000 UTC m=+0.018017058 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T03:12:33.445 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:33 smithi017 podman[92752]: 2024-03-24 03:12:33.218353761 +0000 UTC m=+0.136859956 container create 0dd541a338d1b21c8160e8b849e286fe3df6483fdb8aec76d2cd22346fc9d978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-24T03:12:33.445 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:33 smithi017 podman[92752]: 2024-03-24 03:12:33.446306502 +0000 UTC m=+0.364812699 container init 0dd541a338d1b21c8160e8b849e286fe3df6483fdb8aec76d2cd22346fc9d978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:12:33.547 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:33 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:33.101+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:33.706 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:33 smithi017 podman[92752]: 2024-03-24 03:12:33.449575244 +0000 UTC m=+0.368081431 container start 0dd541a338d1b21c8160e8b849e286fe3df6483fdb8aec76d2cd22346fc9d978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-24T03:12:33.706 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:33 smithi017 podman[92752]: 2024-03-24 03:12:33.546438204 +0000 UTC m=+0.464944396 container attach 0dd541a338d1b21c8160e8b849e286fe3df6483fdb8aec76d2cd22346fc9d978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T03:12:34.047 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:33 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:33.760+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:34.090 INFO:journalctl@ceph.osd.3.smithi017.stdout:Mar 24 03:12:33 smithi017 podman[92752]: 2024-03-24 03:12:33.745202398 +0000 UTC m=+0.663708586 container died 0dd541a338d1b21c8160e8b849e286fe3df6483fdb8aec76d2cd22346fc9d978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-24T03:12:34.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:33 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:33.952+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:34.431 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.3.service' 2024-03-24T03:12:34.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:12:34.476 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-24T03:12:34.476 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-24T03:12:34.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.4 2024-03-24T03:12:34.547 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:34 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:34.070+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:34.807 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:34 smithi019 systemd[1]: Stopping Ceph osd.4 for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:12:35.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:34 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:34.797+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:35.070 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:34 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:34.908+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:35.070 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:34 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4[47074]: 2024-03-24T03:12:34.807+0000 7fb781d51700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:12:35.071 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:34 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4[47074]: 2024-03-24T03:12:34.807+0000 7fb781d51700 -1 osd.4 107 *** Got signal Terminated *** 2024-03-24T03:12:35.071 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:34 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4[47074]: 2024-03-24T03:12:34.807+0000 7fb781d51700 -1 osd.4 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T03:12:35.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:35 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:35.092+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:36.047 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:35 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:35.756+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:36.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:35 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:35.906+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:36.547 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:36 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:36.120+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:37.047 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:36 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:36.780+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:37.310 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:36 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:36.898+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:37.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:37 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:37.171+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:37.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:37 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:37.142+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:37.570 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:37 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4[47074]: 2024-03-24T03:12:37.311+0000 7fb77a33d700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:13.858373+0000 front 2024-03-24T03:12:13.858428+0000 (oldest deadline 2024-03-24T03:12:36.758535+0000) 2024-03-24T03:12:38.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:37 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:37.809+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:38.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:37 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:37.809+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:38.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:37 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:37.946+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:38.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:37 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:37.946+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:38.357 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:38 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:38.217+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:38.358 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:38 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:38.020+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:38.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:38 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:38.132+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:38.820 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:38 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4[47074]: 2024-03-24T03:12:38.358+0000 7fb77a33d700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:13.858373+0000 front 2024-03-24T03:12:13.858428+0000 (oldest deadline 2024-03-24T03:12:36.758535+0000) 2024-03-24T03:12:39.298 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:39 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:39.101+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:39.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:38 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:38.855+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:39.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:38 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:38.855+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:39.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:39 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:39.057+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:39.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:38 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:38.948+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:39.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:38 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:38.948+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:39.369 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:39 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:39.034+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:39.369 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:39 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:39.180+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:39.820 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:39 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4[47074]: 2024-03-24T03:12:39.382+0000 7fb77a33d700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:13.858373+0000 front 2024-03-24T03:12:13.858428+0000 (oldest deadline 2024-03-24T03:12:36.758535+0000) 2024-03-24T03:12:40.123 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:39 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:39.822+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:40.124 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:39 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:39.822+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:40.300 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:40 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:40.043+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:40.300 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:39 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:39.901+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:40.300 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:39 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:39.901+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:40.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:40 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:40.153+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:40.456 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:39 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:39.992+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:40.547 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:40 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:40.123+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:40.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:40 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:40.123+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.604619+0000 front 2024-03-24T03:12:12.604583+0000 (oldest deadline 2024-03-24T03:12:39.605177+0000) 2024-03-24T03:12:40.567 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:40 smithi019 podman[72387]: 2024-03-24 03:12:40.30088254 +0000 UTC m=+5.613070937 container died e59b6e74f5f03d42f4f6592a984c5ae379185bc96bb6ee255dcc327b8338db7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T03:12:40.820 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:40 smithi019 podman[72387]: 2024-03-24 03:12:40.65796431 +0000 UTC m=+5.970152695 container cleanup e59b6e74f5f03d42f4f6592a984c5ae379185bc96bb6ee255dcc327b8338db7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-24T03:12:40.821 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:40 smithi019 bash[72387]: ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4 2024-03-24T03:12:41.144 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:40 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:40.841+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:41.144 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:40 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:40.841+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:41.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:41 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:41.042+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:41.321 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:40 smithi019 podman[72404]: 2024-03-24 03:12:40.967602859 +0000 UTC m=+0.666245444 container remove e59b6e74f5f03d42f4f6592a984c5ae379185bc96bb6ee255dcc327b8338db7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0) 2024-03-24T03:12:41.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:40 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:40.906+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:41.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:40 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:40.906+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:41.458 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:41 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:41.036+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:41.459 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:41 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:41.196+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:41.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:41 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:41.144+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:41.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:41 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:41.144+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.604619+0000 front 2024-03-24T03:12:12.604583+0000 (oldest deadline 2024-03-24T03:12:39.605177+0000) 2024-03-24T03:12:41.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:41 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:41.175+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:41.621 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:41 smithi019 podman[72497]: 2024-03-24 03:12:41.238354205 +0000 UTC m=+0.018268087 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T03:12:41.621 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:41 smithi019 podman[72497]: 2024-03-24 03:12:41.368657943 +0000 UTC m=+0.148571812 container create 1b81f62060dd57a9aabd747fb8c29e88252c7cb9b7594f3cbc517c650d7c9a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T03:12:41.621 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:41 smithi019 podman[72497]: 2024-03-24 03:12:41.621865563 +0000 UTC m=+0.401779447 container init 1b81f62060dd57a9aabd747fb8c29e88252c7cb9b7594f3cbc517c650d7c9a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T03:12:41.925 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:41 smithi019 podman[72497]: 2024-03-24 03:12:41.625683056 +0000 UTC m=+0.405596931 container start 1b81f62060dd57a9aabd747fb8c29e88252c7cb9b7594f3cbc517c650d7c9a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-24T03:12:41.925 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:41 smithi019 podman[72497]: 2024-03-24 03:12:41.721252446 +0000 UTC m=+0.501166322 container attach 1b81f62060dd57a9aabd747fb8c29e88252c7cb9b7594f3cbc517c650d7c9a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux ) 2024-03-24T03:12:42.140 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:41 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:41.880+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:42.140 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:41 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:41.880+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:42.273 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 24 03:12:41 smithi019 podman[72497]: 2024-03-24 03:12:41.925979844 +0000 UTC m=+0.705893724 container died 1b81f62060dd57a9aabd747fb8c29e88252c7cb9b7594f3cbc517c650d7c9a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-24T03:12:42.274 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:41 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:41.940+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:42.274 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:41 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:41.940+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:42.274 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:42 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:42.015+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:42.456 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:42 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:42.021+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:42.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:42 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:42.184+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:42.547 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:42 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:42.160+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:42.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:42 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:42.140+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:42.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:42 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:42.140+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.604619+0000 front 2024-03-24T03:12:12.604583+0000 (oldest deadline 2024-03-24T03:12:39.605177+0000) 2024-03-24T03:12:42.603 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.4.service' 2024-03-24T03:12:42.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:12:42.651 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-24T03:12:42.651 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-24T03:12:42.651 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.5 2024-03-24T03:12:43.014 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:42 smithi188 systemd[1]: Stopping Ceph osd.5 for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:12:43.015 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:42 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:42.868+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:43.015 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:42 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:42.868+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:43.297 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:43 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:43.163+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:43.298 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:43 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:43.014+0000 7fc436711700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:12:43.298 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:43 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:43.014+0000 7fc436711700 -1 osd.5 107 *** Got signal Terminated *** 2024-03-24T03:12:43.298 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:43 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:43.014+0000 7fc436711700 -1 osd.5 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T03:12:43.298 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:43 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:43.149+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:43.298 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:43 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:43.149+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.604619+0000 front 2024-03-24T03:12:12.604583+0000 (oldest deadline 2024-03-24T03:12:39.605177+0000) 2024-03-24T03:12:43.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:42 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:42.975+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:43.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:42 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:42.945+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:43.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:42 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:42.945+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:43.455 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:43 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:43.063+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:43.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:43 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:43.203+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:44.145 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:43 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:43.889+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:44.145 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:43 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:43.889+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:44.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:44.006+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:44.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:43 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:43.969+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:44.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:43 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:43.969+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:44.456 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:44 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:44.046+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:44.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:44 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:44.218+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:44.547 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:44 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:44.154+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:44.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:44 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:44.145+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:44.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:44 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:44.145+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.604619+0000 front 2024-03-24T03:12:12.604583+0000 (oldest deadline 2024-03-24T03:12:39.605177+0000) 2024-03-24T03:12:45.196 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:45 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:45.132+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:45.196 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:45 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:45.132+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.604619+0000 front 2024-03-24T03:12:12.604583+0000 (oldest deadline 2024-03-24T03:12:39.605177+0000) 2024-03-24T03:12:45.196 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:44 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:44.937+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:45.196 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:44 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:44.937+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:45.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:44.966+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:45.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:44.966+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:45.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:44.943+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:45.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:44 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:44.943+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:45.456 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:45 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:45.025+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:45.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:45 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:45.198+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:45.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:45 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:45.198+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:45.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:45 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:45.196+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:45.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:45 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:45.196+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:46.228 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:45 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:45.893+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:46.228 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:45 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:45.893+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:46.228 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:45 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:45.893+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:46.228 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:46 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:46.128+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:46.228 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:46 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:46.128+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.604619+0000 front 2024-03-24T03:12:12.604583+0000 (oldest deadline 2024-03-24T03:12:39.605177+0000) 2024-03-24T03:12:46.228 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:46 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:46.128+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:19.605461+0000 front 2024-03-24T03:12:19.605520+0000 (oldest deadline 2024-03-24T03:12:45.505553+0000) 2024-03-24T03:12:46.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:45 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:45.928+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:46.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:45 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:45.928+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:46.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:45 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:45.982+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:46.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:45 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:45.982+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:46.456 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:45 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:45.985+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:46.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:46 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:46.183+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:46.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:46 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:46.183+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:46.547 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:46 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:46.228+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:46.547 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:46 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:46.228+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:47.105 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:46 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:46.852+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:47.105 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:46 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:46.852+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:47.105 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:46 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:46.852+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:47.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:46 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:46.902+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:47.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:46 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:46.902+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:47.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:46 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:46.982+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:47.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:46 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:46.982+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:47.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:46 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:46.982+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:47.456 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:46 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:46.978+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:47.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:47 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:47.199+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:47.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:47 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:47.199+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:47.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:47 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:47.212+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:47.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:47 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:47.212+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:47.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:47 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:47.105+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:05.103218+0000 front 2024-03-24T03:12:05.103047+0000 (oldest deadline 2024-03-24T03:12:32.103508+0000) 2024-03-24T03:12:47.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:47 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:47.105+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.604619+0000 front 2024-03-24T03:12:12.604583+0000 (oldest deadline 2024-03-24T03:12:39.605177+0000) 2024-03-24T03:12:47.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:47 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5[47871]: 2024-03-24T03:12:47.105+0000 7fc42ecfd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:19.605461+0000 front 2024-03-24T03:12:19.605520+0000 (oldest deadline 2024-03-24T03:12:45.505553+0000) 2024-03-24T03:12:48.222 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:47 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:47.894+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:48.222 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:47 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:47.894+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:48.222 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:47 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:47.894+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:48.227 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:47 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:47.978+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:48.227 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:47 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:47.978+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.368744+0000 front 2024-03-24T03:12:22.368829+0000 (oldest deadline 2024-03-24T03:12:47.068601+0000) 2024-03-24T03:12:48.228 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:48 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:48.228+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:48.228 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:48 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:48.228+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:48.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:47 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:47.919+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:48.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:47 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:47.919+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:48.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:47 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:47.957+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:48.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:47 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:47.957+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:48.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:47 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:47.957+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:48.548 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:48 smithi188 podman[72982]: 2024-03-24 03:12:48.388815335 +0000 UTC m=+5.531322084 container died b2794db016f98e07b18fccca5143503805ea0bfaa3f5507a90e820ea7c2e41f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-24T03:12:48.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:48 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:48.222+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:48.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:48 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:48.222+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:49.039 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:48 smithi188 podman[72982]: 2024-03-24 03:12:48.781177954 +0000 UTC m=+5.923684694 container cleanup b2794db016f98e07b18fccca5143503805ea0bfaa3f5507a90e820ea7c2e41f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T03:12:49.040 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:48 smithi188 bash[72982]: ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5 2024-03-24T03:12:49.040 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:48 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:48.895+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:49.040 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:48 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:48.895+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:49.040 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:48 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:48.896+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:49.205 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:48 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:48.931+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:49.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:48 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:48.931+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.368744+0000 front 2024-03-24T03:12:22.368829+0000 (oldest deadline 2024-03-24T03:12:47.068601+0000) 2024-03-24T03:12:49.297 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:49 smithi188 podman[73002]: 2024-03-24 03:12:49.040784001 +0000 UTC m=+0.645644384 container remove b2794db016f98e07b18fccca5143503805ea0bfaa3f5507a90e820ea7c2e41f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS) 2024-03-24T03:12:49.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:49 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:49.196+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:49.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:49 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:49.196+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:49.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:48 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:48.898+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:49.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:48 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:48.898+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:49.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:49 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:49.001+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:49.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:49 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:49.001+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:49.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:49 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:49.001+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:49.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:49 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:49.250+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:49.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:49 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:49.250+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:49.710 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:49 smithi188 podman[73096]: 2024-03-24 03:12:49.290440104 +0000 UTC m=+0.017611932 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T03:12:49.710 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:49 smithi188 podman[73096]: 2024-03-24 03:12:49.408564878 +0000 UTC m=+0.135736696 container create 3cbb255eeb6438c7b9c271058d86dc2bc6533841f23ccc3c2c1fa6ed1a896a97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-24T03:12:49.710 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:49 smithi188 podman[73096]: 2024-03-24 03:12:49.619339046 +0000 UTC m=+0.346510864 container init 3cbb255eeb6438c7b9c271058d86dc2bc6533841f23ccc3c2c1fa6ed1a896a97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-03-24T03:12:49.710 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:49 smithi188 podman[73096]: 2024-03-24 03:12:49.622588253 +0000 UTC m=+0.349760068 container start 3cbb255eeb6438c7b9c271058d86dc2bc6533841f23ccc3c2c1fa6ed1a896a97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-24T03:12:49.711 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:49 smithi188 podman[73096]: 2024-03-24 03:12:49.71088217 +0000 UTC m=+0.438054001 container attach 3cbb255eeb6438c7b9c271058d86dc2bc6533841f23ccc3c2c1fa6ed1a896a97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-24T03:12:50.048 INFO:journalctl@ceph.osd.5.smithi188.stdout:Mar 24 03:12:49 smithi188 podman[73096]: 2024-03-24 03:12:49.916777959 +0000 UTC m=+0.643949784 container died 3cbb255eeb6438c7b9c271058d86dc2bc6533841f23ccc3c2c1fa6ed1a896a97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-24T03:12:50.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:49 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:49.901+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:50.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:49 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:49.901+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:50.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:49 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:49.901+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:50.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:49 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:49.908+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:50.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:49 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:49.908+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.368744+0000 front 2024-03-24T03:12:22.368829+0000 (oldest deadline 2024-03-24T03:12:47.068601+0000) 2024-03-24T03:12:50.221 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:49 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:49.916+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:50.221 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:49 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:49.916+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:50.221 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:50.024+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:50.221 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:50.024+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:50.221 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:50.024+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:50.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:50.202+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:50.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:50.202+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:50.597 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.5.service' 2024-03-24T03:12:50.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:12:50.639 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-24T03:12:50.639 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-24T03:12:50.639 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.6 2024-03-24T03:12:50.669 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:50.292+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:50.669 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:50.292+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:50.952 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:50 smithi017 systemd[1]: Stopping Ceph osd.6 for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:12:50.952 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:50.905+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:50.952 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:50.905+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.368744+0000 front 2024-03-24T03:12:22.368829+0000 (oldest deadline 2024-03-24T03:12:47.068601+0000) 2024-03-24T03:12:51.205 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:50.953+0000 7f2efdf77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:12:51.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:50.953+0000 7f2efdf77700 -1 osd.6 107 *** Got signal Terminated *** 2024-03-24T03:12:51.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:50 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:50.953+0000 7f2efdf77700 -1 osd.6 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T03:12:51.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:51 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:51.177+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:51.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:51 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:51.177+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:51.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:50.949+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:51.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:50.949+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:51.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:50 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:50.949+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:51.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:51 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:51.000+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:51.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:51 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:51.000+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:51.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:51 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:51.000+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:51.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:50.954+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:51.321 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:50 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:50.954+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:51.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:51 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:51.324+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:51.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:51 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:51.324+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:52.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:51 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:51.941+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:52.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:51 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:51.941+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.368744+0000 front 2024-03-24T03:12:22.368829+0000 (oldest deadline 2024-03-24T03:12:47.068601+0000) 2024-03-24T03:12:52.227 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:51 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:51.909+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:52.227 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:51 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:51.909+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:52.227 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:51 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:51.909+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:52.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:51 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:51.958+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:52.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:51 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:51.958+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:52.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:52 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:52.035+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:52.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:52 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:52.035+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:52.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:52 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:52.035+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:52.547 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:52 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:52.227+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:52.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:52 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:52.227+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:52.705 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:52 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:52.317+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:52.705 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:52 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:52.317+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:53.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:52 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:52.898+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:53.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:52 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:52.898+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.368744+0000 front 2024-03-24T03:12:22.368829+0000 (oldest deadline 2024-03-24T03:12:47.068601+0000) 2024-03-24T03:12:53.257 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:52 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:52.927+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:53.257 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:52 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:52.927+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:53.257 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:52 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:52.927+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:53.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:52 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:52.940+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:53.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:52 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:52.940+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:53.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:52 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:52.940+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:30.677991+0000 front 2024-03-24T03:12:30.678097+0000 (oldest deadline 2024-03-24T03:12:52.378148+0000) 2024-03-24T03:12:53.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:53 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:53.055+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:53.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:53 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:53.055+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:53.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:53 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:53.055+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:53.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:53 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:53.055+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:12:53.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:53 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:53.257+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:53.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:53 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:53.257+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:53.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:53 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:53.337+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:53.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:53 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:53.337+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:54.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:53 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:53.855+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:54.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:53 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:53.855+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.368744+0000 front 2024-03-24T03:12:22.368829+0000 (oldest deadline 2024-03-24T03:12:47.068601+0000) 2024-03-24T03:12:54.296 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:53 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:53.910+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:54.296 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:53 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:53.910+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:54.296 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:53 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:53.910+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:54.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:53 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:53.894+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:54.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:53 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:53.894+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:54.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:53 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:53.894+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:30.677991+0000 front 2024-03-24T03:12:30.678097+0000 (oldest deadline 2024-03-24T03:12:52.378148+0000) 2024-03-24T03:12:54.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:54 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:54.041+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:54.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:54 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:54.041+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:54.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:54 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:54.041+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:54.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:54 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:54.041+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:12:54.547 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:54 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:54.296+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:54.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:54 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:54.296+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:54.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:54 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:54.310+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:54.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:54 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:54.310+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:55.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:54 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:54.860+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:55.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:54 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:54.860+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.368744+0000 front 2024-03-24T03:12:22.368829+0000 (oldest deadline 2024-03-24T03:12:47.068601+0000) 2024-03-24T03:12:55.289 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:54 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:54.956+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:55.289 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:54 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:54.956+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:55.289 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:54 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:54.956+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:55.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:54 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:54.881+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:55.321 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:54 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:54.881+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:55.321 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:54 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:54.881+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:30.677991+0000 front 2024-03-24T03:12:30.678097+0000 (oldest deadline 2024-03-24T03:12:52.378148+0000) 2024-03-24T03:12:55.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:55 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:55.084+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:55.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:55 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:55.084+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:55.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:55 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:55.084+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:55.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:55 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:55.084+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:12:55.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:55 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:55.288+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:55.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:55 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:55.288+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:55.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:55 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:55.341+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:55.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:55 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:55.341+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:55.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:55 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:55.341+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:12:56.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:55 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:55.869+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.466899+0000 front 2024-03-24T03:12:12.466968+0000 (oldest deadline 2024-03-24T03:12:37.166909+0000) 2024-03-24T03:12:56.206 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:55 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6[59943]: 2024-03-24T03:12:55.869+0000 7f2efa56b700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.368744+0000 front 2024-03-24T03:12:22.368829+0000 (oldest deadline 2024-03-24T03:12:47.068601+0000) 2024-03-24T03:12:56.270 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:55 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:55.958+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:56.270 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:55 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:55.958+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:56.270 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:55 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:55.958+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:56.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:55 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:55.928+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:56.321 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:55 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:55.928+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:56.321 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:55 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:55.928+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:30.677991+0000 front 2024-03-24T03:12:30.678097+0000 (oldest deadline 2024-03-24T03:12:52.378148+0000) 2024-03-24T03:12:56.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:56 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:56.043+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:56.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:56 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:56.043+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:56.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:56 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:56.043+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:56.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:56 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:56.043+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:12:56.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:56 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:56.270+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:56.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:56 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:56.270+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:56.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:56 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:56.270+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:12:56.603 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:56 smithi017 podman[92870]: 2024-03-24 03:12:56.30215564 +0000 UTC m=+5.462562491 container died 9480c05fcace8e7a7f107d57f0f8c8741a483d0b57e2ced277333c7f5195ea03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS) 2024-03-24T03:12:56.603 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:56.388+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:56.603 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:56.388+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:56.604 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:56 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:56.388+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:12:56.956 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:56 smithi017 podman[92870]: 2024-03-24 03:12:56.703212076 +0000 UTC m=+5.863618914 container cleanup 9480c05fcace8e7a7f107d57f0f8c8741a483d0b57e2ced277333c7f5195ea03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-24T03:12:56.956 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:56 smithi017 bash[92870]: ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6 2024-03-24T03:12:57.232 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:56 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:56.981+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:57.232 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:56 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:56.981+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:57.232 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:56 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:56.981+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:57.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:56 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:56.920+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:57.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:56 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:56.920+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:57.321 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:56 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:56.920+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:30.677991+0000 front 2024-03-24T03:12:30.678097+0000 (oldest deadline 2024-03-24T03:12:52.378148+0000) 2024-03-24T03:12:57.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:57 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:57.061+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:57.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:57 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:57.061+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:57.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:57 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:57.061+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:57.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:57 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:57.061+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:12:57.425 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:57 smithi017 podman[92887]: 2024-03-24 03:12:57.096376822 +0000 UTC m=+0.789244389 container remove 9480c05fcace8e7a7f107d57f0f8c8741a483d0b57e2ced277333c7f5195ea03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-24T03:12:57.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:57 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:57.232+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:57.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:57 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:57.232+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:57.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:57 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:57.232+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:12:57.706 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:57 smithi017 podman[92976]: 2024-03-24 03:12:57.410291484 +0000 UTC m=+0.017898233 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T03:12:57.706 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:57 smithi017 podman[92976]: 2024-03-24 03:12:57.530655573 +0000 UTC m=+0.138262357 container create cadfcf572d2a99537b17c68c510acca0d8c5462233feaf855d00789d2e998286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-24T03:12:57.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:57.426+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:57.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:57.426+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:57.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:57 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:57.426+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:12:58.186 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:57 smithi017 podman[92976]: 2024-03-24 03:12:57.882994886 +0000 UTC m=+0.490601636 container init cadfcf572d2a99537b17c68c510acca0d8c5462233feaf855d00789d2e998286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-24T03:12:58.186 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:57 smithi017 podman[92976]: 2024-03-24 03:12:57.886086553 +0000 UTC m=+0.493693295 container start cadfcf572d2a99537b17c68c510acca0d8c5462233feaf855d00789d2e998286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-24T03:12:58.187 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:57 smithi017 podman[92976]: 2024-03-24 03:12:57.983052001 +0000 UTC m=+0.590658745 container attach cadfcf572d2a99537b17c68c510acca0d8c5462233feaf855d00789d2e998286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-24T03:12:58.187 INFO:journalctl@ceph.osd.6.smithi017.stdout:Mar 24 03:12:58 smithi017 podman[92976]: 2024-03-24 03:12:58.187378807 +0000 UTC m=+0.794985540 container died cadfcf572d2a99537b17c68c510acca0d8c5462233feaf855d00789d2e998286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:12:58.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:58 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:58.191+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:58.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:58 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:58.191+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:58.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:58 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:58.191+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:12:58.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:57 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:57.968+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:58.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:57 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:57.968+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:58.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:57 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:57.968+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:58.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:57 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:57.968+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:58.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:57 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:57.968+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:58.321 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:57 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:57.968+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:30.677991+0000 front 2024-03-24T03:12:30.678097+0000 (oldest deadline 2024-03-24T03:12:52.378148+0000) 2024-03-24T03:12:58.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:58.071+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:58.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:58.071+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:58.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:58.071+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:58.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:58.071+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:12:58.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:58 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:58.389+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:58.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:58 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:58.389+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:58.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:58 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:58.389+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:12:58.892 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.6.service' 2024-03-24T03:12:58.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:12:58.978 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-24T03:12:58.978 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-24T03:12:58.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.7 2024-03-24T03:12:59.295 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:59.044+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:12:59.295 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:59.044+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:12:59.295 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:59.044+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:12:59.295 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:12:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:12:59.044+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:12:59.296 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:58.984+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:12:59.296 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:58.984+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:12:59.296 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:58 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:58.984+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:30.677991+0000 front 2024-03-24T03:12:30.678097+0000 (oldest deadline 2024-03-24T03:12:52.378148+0000) 2024-03-24T03:12:59.296 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:59 smithi019 systemd[1]: Stopping Ceph osd.7 for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:12:59.296 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:59.296+0000 7fcc4eeae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:12:59.296 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:59.296+0000 7fcc4eeae700 -1 osd.7 107 *** Got signal Terminated *** 2024-03-24T03:12:59.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:59.202+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:12:59.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:59.202+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:12:59.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:12:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:12:59.202+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:12:59.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:59.014+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:12:59.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:59.014+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:12:59.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:59.014+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:12:59.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:59 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:59.412+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:12:59.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:59 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:59.412+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:12:59.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:12:59 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:12:59.412+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:00.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:00 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:00.159+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:13:00.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:00 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:00.159+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:13:00.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:00 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:00.159+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:13:00.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:59.974+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:00.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:59.974+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:00.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:12:59 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:12:59.974+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:00.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:00 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:00.067+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:00.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:00 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:00.067+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:00.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:00 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:00.067+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:00.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:00 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:00.067+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:00.321 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:59.296+0000 7fcc4eeae700 -1 osd.7 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T03:13:00.322 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:59.968+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:13:00.322 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:59.968+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:13:00.322 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:12:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:12:59.968+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:30.677991+0000 front 2024-03-24T03:12:30.678097+0000 (oldest deadline 2024-03-24T03:12:52.378148+0000) 2024-03-24T03:13:00.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:00.433+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:00.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:00.433+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:00.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:00 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:00.433+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:01.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:01 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:01.142+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:13:01.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:01 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:01.142+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:13:01.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:01 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:01.142+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:13:01.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:00 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:00.929+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:01.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:00 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:00.929+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:01.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:00 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:00.929+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:01.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:00 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:13:00.918+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:13:01.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:00 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:13:00.918+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:13:01.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:00 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:13:00.918+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:30.677991+0000 front 2024-03-24T03:12:30.678097+0000 (oldest deadline 2024-03-24T03:12:52.378148+0000) 2024-03-24T03:13:01.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:01 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:01.114+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:01.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:01 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:01.114+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:01.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:01 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:01.114+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:01.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:01 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:01.114+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:01.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:01 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:01.399+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:01.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:01 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:01.399+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:01.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:01 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:01.399+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:01.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:01 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:01.399+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:02.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:02 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:02.137+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:13:02.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:02 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:02.137+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:13:02.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:02 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:02.137+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:13:02.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:02 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:02.137+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.632637+0000 front 2024-03-24T03:12:35.632573+0000 (oldest deadline 2024-03-24T03:13:01.532466+0000) 2024-03-24T03:13:02.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:01 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:01.927+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:02.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:01 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:01.927+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:02.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:01 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:01.927+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:02.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:01 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:01.927+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:02.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:01 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:13:01.894+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:13:02.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:01 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:13:01.894+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:13:02.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:01 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:13:01.894+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:30.677991+0000 front 2024-03-24T03:12:30.678097+0000 (oldest deadline 2024-03-24T03:12:52.378148+0000) 2024-03-24T03:13:02.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:02.100+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:02.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:02.100+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:02.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:02.100+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:02.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:02.100+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:02.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:02.100+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:02.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:02 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:02.363+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:02.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:02 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:02.363+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:02.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:02 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:02.363+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:02.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:02 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:02.363+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:03.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:03 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:03.140+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:13:03.300 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:03 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:03.140+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:13:03.300 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:03 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:03.140+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:13:03.300 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:03 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:03.140+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.632637+0000 front 2024-03-24T03:12:35.632573+0000 (oldest deadline 2024-03-24T03:13:01.532466+0000) 2024-03-24T03:13:03.301 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:02 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:02.892+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:03.301 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:02 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:02.892+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:03.301 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:02 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:02.892+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:03.301 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:02 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:02.892+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:03.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:13:02.886+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:13:03.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:13:02.886+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:13:03.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:02 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:13:02.886+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:30.677991+0000 front 2024-03-24T03:12:30.678097+0000 (oldest deadline 2024-03-24T03:12:52.378148+0000) 2024-03-24T03:13:03.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:03 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:03.071+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:03.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:03 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:03.071+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:03.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:03 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:03.071+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:03.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:03 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:03.071+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:03.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:03 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:03.071+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:03.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:03 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:03.369+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:03.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:03 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:03.369+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:03.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:03 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:03.369+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:03.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:03 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:03.369+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:04.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:04 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:04.124+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:13:04.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:04 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:04.124+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:13:04.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:04 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:04.124+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:13:04.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:04 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:04.124+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.632637+0000 front 2024-03-24T03:12:35.632573+0000 (oldest deadline 2024-03-24T03:13:01.532466+0000) 2024-03-24T03:13:04.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:03 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:03.894+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:04.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:03 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:03.894+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:04.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:03 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:03.894+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:04.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:03 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:03.894+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:04.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:03 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:13:03.869+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.474088+0000 front 2024-03-24T03:12:14.474122+0000 (oldest deadline 2024-03-24T03:12:38.474891+0000) 2024-03-24T03:13:04.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:03 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:13:03.869+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.976386+0000 front 2024-03-24T03:12:21.976302+0000 (oldest deadline 2024-03-24T03:12:44.876236+0000) 2024-03-24T03:13:04.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:03 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7[54070]: 2024-03-24T03:13:03.869+0000 7fcc4b4a2700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:30.677991+0000 front 2024-03-24T03:12:30.678097+0000 (oldest deadline 2024-03-24T03:12:52.378148+0000) 2024-03-24T03:13:04.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:04 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:04.077+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:04.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:04 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:04.077+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:04.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:04 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:04.077+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:04.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:04 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:04.077+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:04.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:04 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:04.077+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:04.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:04.372+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:04.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:04.372+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:04.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:04.372+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:04.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:04 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:04.372+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:04.970 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:04 smithi019 podman[72615]: 2024-03-24 03:13:04.649160847 +0000 UTC m=+5.455854660 container died e47d8ce0b2a3d9e80a5818528f485f1f2df4489e62390bd8ef1060b18ecb46d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T03:13:05.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:04 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:04.940+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:05.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:04 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:04.940+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:05.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:04 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:04.940+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:05.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:04 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:04.940+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:05.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:05 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:05.080+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:13:05.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:05 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:05.080+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:13:05.299 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:05 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:05.080+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:13:05.299 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:05 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:05.080+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.632637+0000 front 2024-03-24T03:12:35.632573+0000 (oldest deadline 2024-03-24T03:13:01.532466+0000) 2024-03-24T03:13:05.304 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:05 smithi019 podman[72615]: 2024-03-24 03:13:05.053716372 +0000 UTC m=+5.860410177 container cleanup e47d8ce0b2a3d9e80a5818528f485f1f2df4489e62390bd8ef1060b18ecb46d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-24T03:13:05.304 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:05 smithi019 bash[72615]: ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7 2024-03-24T03:13:05.304 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:05 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:05.034+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:05.305 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:05 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:05.034+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:05.305 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:05 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:05.034+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:05.305 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:05 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:05.034+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:05.305 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:05 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:05.034+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:05.570 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:05 smithi019 podman[72633]: 2024-03-24 03:13:05.305052759 +0000 UTC m=+0.649706138 container remove e47d8ce0b2a3d9e80a5818528f485f1f2df4489e62390bd8ef1060b18ecb46d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-24T03:13:05.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:05 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:05.396+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:05.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:05 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:05.396+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:05.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:05 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:05.396+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:05.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:05 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:05.396+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:05.957 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:05 smithi019 podman[72726]: 2024-03-24 03:13:05.527121984 +0000 UTC m=+0.018502621 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T03:13:05.957 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:05 smithi019 podman[72726]: 2024-03-24 03:13:05.631066459 +0000 UTC m=+0.122447143 container create fd61e22041c504e1e9c283f1413adbad1f4ec57fae065ed9fccb01ddb8bfac8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS) 2024-03-24T03:13:05.958 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:05 smithi019 podman[72726]: 2024-03-24 03:13:05.858621032 +0000 UTC m=+0.350001647 container init fd61e22041c504e1e9c283f1413adbad1f4ec57fae065ed9fccb01ddb8bfac8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:13:05.958 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:05 smithi019 podman[72726]: 2024-03-24 03:13:05.861784119 +0000 UTC m=+0.353164732 container start fd61e22041c504e1e9c283f1413adbad1f4ec57fae065ed9fccb01ddb8bfac8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-24T03:13:05.958 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:05 smithi019 podman[72726]: 2024-03-24 03:13:05.958497707 +0000 UTC m=+0.449878338 container attach fd61e22041c504e1e9c283f1413adbad1f4ec57fae065ed9fccb01ddb8bfac8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-24T03:13:06.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:06 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:06.113+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:13:06.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:06 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:06.113+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:13:06.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:06 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:06.113+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:13:06.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:06 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:06.113+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.632637+0000 front 2024-03-24T03:12:35.632573+0000 (oldest deadline 2024-03-24T03:13:01.532466+0000) 2024-03-24T03:13:06.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:05 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:05.919+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:06.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:05 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:05.919+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:06.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:05 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:05.919+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:06.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:05 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:05.919+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:06.320 INFO:journalctl@ceph.osd.7.smithi019.stdout:Mar 24 03:13:06 smithi019 podman[72726]: 2024-03-24 03:13:06.160291796 +0000 UTC m=+0.651672425 container died fd61e22041c504e1e9c283f1413adbad1f4ec57fae065ed9fccb01ddb8bfac8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-24T03:13:06.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:06 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:06.026+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:06.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:06 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:06.026+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:06.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:06 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:06.026+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:06.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:06 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:06.026+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:06.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:06 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:06.026+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:06.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:06 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:06.428+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:06.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:06 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:06.428+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:06.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:06 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:06.428+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:06.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:06 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:06.428+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:06.787 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.7.service' 2024-03-24T03:13:06.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:13:06.832 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-24T03:13:06.832 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-24T03:13:06.832 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.8 2024-03-24T03:13:07.127 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:06 smithi188 systemd[1]: Stopping Ceph osd.8 for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:13:07.127 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:06 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:06.874+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:07.128 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:06 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:06.874+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:07.128 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:06 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:06.874+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:07.128 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:06 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:06.874+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:07.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:07 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:07.066+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:07.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:07 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:07.066+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:07.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:07 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:07.066+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:07.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:07 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:07.066+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:07.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:07 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:07.066+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:07.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:07.127+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:13:07.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:07.127+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:13:07.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:07.127+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:13:07.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:07.127+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.632637+0000 front 2024-03-24T03:12:35.632573+0000 (oldest deadline 2024-03-24T03:13:01.532466+0000) 2024-03-24T03:13:07.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:07.170+0000 7fd6e1af3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:13:07.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:07.170+0000 7fd6e1af3700 -1 osd.8 107 *** Got signal Terminated *** 2024-03-24T03:13:07.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:07.170+0000 7fd6e1af3700 -1 osd.8 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T03:13:07.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:07.410+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:07.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:07.410+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:07.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:07.410+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:07.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:07 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:07.410+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:08.159 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:07.825+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:08.159 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:07.825+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:08.159 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:07.825+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:08.159 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:07 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:07.825+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:08.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:08 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:08.018+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:08.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:08 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:08.018+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:08.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:08 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:08.018+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:08.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:08 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:08.018+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:08.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:08 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:08.018+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:08.547 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:08 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:08.159+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:13:08.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:08 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:08.159+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:13:08.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:08 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:08.159+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:13:08.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:08 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:08.159+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.632637+0000 front 2024-03-24T03:12:35.632573+0000 (oldest deadline 2024-03-24T03:13:01.532466+0000) 2024-03-24T03:13:08.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:08 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:08.389+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:08.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:08 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:08.389+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:08.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:08 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:08.389+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:08.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:08 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:08.389+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:09.161 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:08 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:08.827+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:09.161 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:08 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:08.827+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:09.161 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:08 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:08.827+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:09.162 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:08 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:08.827+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:09.162 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:08 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:08.827+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:09.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:08 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:08.983+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:09.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:08 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:08.983+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:09.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:08 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:08.983+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:09.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:08 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:08.983+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:09.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:08 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:08.983+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:09.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:09 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:09.161+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:13:09.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:09 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:09.161+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:13:09.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:09 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:09.161+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:13:09.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:09 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:09.161+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.632637+0000 front 2024-03-24T03:12:35.632573+0000 (oldest deadline 2024-03-24T03:13:01.532466+0000) 2024-03-24T03:13:09.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:09 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:09.161+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.933593+0000 front 2024-03-24T03:12:47.933636+0000 (oldest deadline 2024-03-24T03:13:08.433744+0000) 2024-03-24T03:13:09.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:09 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:09.403+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:09.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:09 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:09.403+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:09.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:09 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:09.403+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:09.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:09 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:09.403+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:09.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:09 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:09.403+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:43.935289+0000 front 2024-03-24T03:12:43.935404+0000 (oldest deadline 2024-03-24T03:13:09.235228+0000) 2024-03-24T03:13:10.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:09 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:09.865+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:10.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:09 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:09.865+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:10.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:09 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:09.865+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:10.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:09 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:09.865+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:10.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:09 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:09.865+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:10.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:09 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:09.961+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:10.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:09 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:09.961+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:10.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:09 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:09.961+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:10.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:09 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:09.961+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:10.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:09 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:09.961+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:10.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:09 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:09.961+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:10.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:10 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:10.180+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:13:10.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:10 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:10.180+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:13:10.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:10 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:10.180+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:13:10.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:10 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:10.180+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.632637+0000 front 2024-03-24T03:12:35.632573+0000 (oldest deadline 2024-03-24T03:13:01.532466+0000) 2024-03-24T03:13:10.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:10 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:10.180+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.933593+0000 front 2024-03-24T03:12:47.933636+0000 (oldest deadline 2024-03-24T03:13:08.433744+0000) 2024-03-24T03:13:10.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:10 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:10.435+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:10.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:10 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:10.435+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:10.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:10 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:10.435+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:10.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:10 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:10.435+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:10.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:10 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:10.435+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:43.935289+0000 front 2024-03-24T03:12:43.935404+0000 (oldest deadline 2024-03-24T03:13:09.235228+0000) 2024-03-24T03:13:11.188 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:10 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:10.833+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:11.189 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:10 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:10.833+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:11.189 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:10 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:10.833+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:11.189 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:10 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:10.833+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:11.189 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:10 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:10.833+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:11.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:10 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:10.994+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:11.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:10 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:10.994+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:11.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:10 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:10.994+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:11.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:10 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:10.994+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:11.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:10 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:10.994+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:11.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:10 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:10.994+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:11.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:11 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:11.189+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.829399+0000 front 2024-03-24T03:12:15.829346+0000 (oldest deadline 2024-03-24T03:12:40.429794+0000) 2024-03-24T03:13:11.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:11 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:11.189+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:20.430149+0000 front 2024-03-24T03:12:20.430235+0000 (oldest deadline 2024-03-24T03:12:44.530313+0000) 2024-03-24T03:13:11.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:11 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:11.189+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.732115+0000 front 2024-03-24T03:12:29.732089+0000 (oldest deadline 2024-03-24T03:12:55.632022+0000) 2024-03-24T03:13:11.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:11 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:11.189+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.632637+0000 front 2024-03-24T03:12:35.632573+0000 (oldest deadline 2024-03-24T03:13:01.532466+0000) 2024-03-24T03:13:11.548 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:11 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8[54608]: 2024-03-24T03:13:11.189+0000 7fd6da0df700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.933593+0000 front 2024-03-24T03:12:47.933636+0000 (oldest deadline 2024-03-24T03:13:08.433744+0000) 2024-03-24T03:13:11.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:11 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:11.468+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:11.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:11 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:11.468+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:11.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:11 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:11.468+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:11.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:11 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:11.468+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:11.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:11 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:11.468+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:43.935289+0000 front 2024-03-24T03:12:43.935404+0000 (oldest deadline 2024-03-24T03:13:09.235228+0000) 2024-03-24T03:13:12.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:11 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:11.856+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:12.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:11 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:11.856+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:12.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:11 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:11.856+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:12.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:11 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:11.856+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:12.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:11 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:11.856+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:12.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:11 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:11.973+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:12.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:11 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:11.973+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:12.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:11 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:11.973+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:12.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:11 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:11.973+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:12.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:11 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:11.973+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:12.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:11 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:11.973+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:12.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:12 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:12.446+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:12.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:12 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:12.446+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:12.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:12 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:12.446+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:12.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:12 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:12.446+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:12.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:12 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:12.446+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:43.935289+0000 front 2024-03-24T03:12:43.935404+0000 (oldest deadline 2024-03-24T03:13:09.235228+0000) 2024-03-24T03:13:12.915 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:12 smithi188 podman[73214]: 2024-03-24 03:13:12.647441091 +0000 UTC m=+5.623791422 container died 068c762b871d7ba1c947c11447180eff53342bcf05d00f404d241b331ddd1c4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-24T03:13:12.915 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:12 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:12.856+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:12.916 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:12 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:12.856+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:12.916 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:12 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:12.856+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:12.916 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:12 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:12.856+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:12.916 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:12 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:12.856+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:13.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:13 smithi188 podman[73214]: 2024-03-24 03:13:13.023641513 +0000 UTC m=+5.999991895 container cleanup 068c762b871d7ba1c947c11447180eff53342bcf05d00f404d241b331ddd1c4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-24T03:13:13.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:13 smithi188 bash[73214]: ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8 2024-03-24T03:13:13.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:12 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:12.973+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:13.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:12 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:12.973+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:13.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:12 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:12.973+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:13.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:12 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:12.973+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:13.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:12 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:12.973+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:13.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:12 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:12.973+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:13.629 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:13 smithi188 podman[73232]: 2024-03-24 03:13:13.316366537 +0000 UTC m=+0.663371072 container remove 068c762b871d7ba1c947c11447180eff53342bcf05d00f404d241b331ddd1c4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:13:13.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:13 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:13.486+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:13.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:13 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:13.486+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:13.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:13 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:13.486+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:13.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:13 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:13.486+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:13.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:13 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:13.486+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:43.935289+0000 front 2024-03-24T03:12:43.935404+0000 (oldest deadline 2024-03-24T03:13:09.235228+0000) 2024-03-24T03:13:13.960 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:13 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:13.858+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:13.960 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:13 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:13.858+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:13.960 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:13 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:13.858+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:13.960 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:13 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:13.858+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:13.961 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:13 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:13.858+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:13.961 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:13 smithi188 podman[73324]: 2024-03-24 03:13:13.533225344 +0000 UTC m=+0.017697971 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T03:13:13.961 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:13 smithi188 podman[73324]: 2024-03-24 03:13:13.64224268 +0000 UTC m=+0.126715346 container create 214ab91a1490f616ec5e67eeef17428b009dbbf9d1782ca5877a2b94b5379655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T03:13:13.961 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:13 smithi188 podman[73324]: 2024-03-24 03:13:13.86961426 +0000 UTC m=+0.354086874 container init 214ab91a1490f616ec5e67eeef17428b009dbbf9d1782ca5877a2b94b5379655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T03:13:13.961 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:13 smithi188 podman[73324]: 2024-03-24 03:13:13.872650688 +0000 UTC m=+0.357123302 container start 214ab91a1490f616ec5e67eeef17428b009dbbf9d1782ca5877a2b94b5379655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-24T03:13:13.961 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:13 smithi188 podman[73324]: 2024-03-24 03:13:13.961585163 +0000 UTC m=+0.446057791 container attach 214ab91a1490f616ec5e67eeef17428b009dbbf9d1782ca5877a2b94b5379655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-24T03:13:14.298 INFO:journalctl@ceph.osd.8.smithi188.stdout:Mar 24 03:13:14 smithi188 podman[73324]: 2024-03-24 03:13:14.154745113 +0000 UTC m=+0.639217728 container died 214ab91a1490f616ec5e67eeef17428b009dbbf9d1782ca5877a2b94b5379655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-24T03:13:14.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:14 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:14.016+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:14.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:14 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:14.016+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:14.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:14 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:14.016+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:14.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:14 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:14.016+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:14.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:14 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:14.016+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:14.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:14 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:14.016+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:14.749 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.8.service' 2024-03-24T03:13:14.791 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:13:14.791 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-24T03:13:14.791 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-24T03:13:14.791 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.9 2024-03-24T03:13:14.820 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:14 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:14.517+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:14.820 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:14 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:14.517+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:14.820 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:14 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:14.517+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:14.820 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:14 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:14.517+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:14.821 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:14 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:14.517+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:43.935289+0000 front 2024-03-24T03:12:43.935404+0000 (oldest deadline 2024-03-24T03:13:09.235228+0000) 2024-03-24T03:13:15.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:14 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:14.868+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:15.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:14 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:14.868+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:15.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:14 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:14.868+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:15.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:14 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:14.868+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:15.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:14 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:14.868+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:15.088 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:14 smithi017 systemd[1]: Stopping Ceph osd.9 for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:13:15.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:15 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:15.064+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:15.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:15 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:15.064+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:15.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:15 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:15.064+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:15.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:15 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:15.064+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:15.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:15 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:15.064+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:15.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:15 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:15.064+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:15.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:15 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:15.089+0000 7f78d9d4e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:13:15.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:15 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:15.089+0000 7f78d9d4e700 -1 osd.9 107 *** Got signal Terminated *** 2024-03-24T03:13:15.456 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:15 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:15.089+0000 7f78d9d4e700 -1 osd.9 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T03:13:15.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:15 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:15.527+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:15.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:15 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:15.527+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:15.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:15 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:15.527+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:15.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:15 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:15.527+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:15.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:15 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:15.527+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:43.935289+0000 front 2024-03-24T03:12:43.935404+0000 (oldest deadline 2024-03-24T03:13:09.235228+0000) 2024-03-24T03:13:16.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:15 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:15.859+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:16.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:15 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:15.859+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:16.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:15 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:15.859+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:16.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:15 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:15.859+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:16.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:15 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:15.859+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:16.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:16 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:16.037+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:16.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:16 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:16.037+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:16.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:16 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:16.037+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:16.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:16 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:16.037+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:16.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:16 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:16.037+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:16.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:16 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:16.037+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:16.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:16 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:16.521+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:16.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:16 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:16.521+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:16.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:16 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:16.521+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:16.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:16 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:16.521+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:16.957 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:16 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:16.521+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:43.935289+0000 front 2024-03-24T03:12:43.935404+0000 (oldest deadline 2024-03-24T03:13:09.235228+0000) 2024-03-24T03:13:17.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:16 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:16.885+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:17.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:16 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:16.885+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:17.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:16 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:16.885+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:17.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:16 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:16.885+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:17.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:16 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:16.885+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:17.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:17 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:17.006+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:17.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:17 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:17.006+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:17.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:17 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:17.006+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:17.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:17 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:17.006+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:17.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:17 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:17.006+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:17.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:17 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:17.006+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:17.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:17 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:17.006+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.913269+0000 front 2024-03-24T03:12:54.913360+0000 (oldest deadline 2024-03-24T03:13:16.613232+0000) 2024-03-24T03:13:17.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:17 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:17.490+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:17.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:17 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:17.490+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:17.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:17 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:17.490+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:17.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:17 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:17.490+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:17.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:17 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:17.490+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:43.935289+0000 front 2024-03-24T03:12:43.935404+0000 (oldest deadline 2024-03-24T03:13:09.235228+0000) 2024-03-24T03:13:18.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:17 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:17.880+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:18.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:17 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:17.880+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:18.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:17 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:17.880+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:18.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:17 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:17.880+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:18.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:17 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:17.880+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:18.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:18 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:18.033+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:18.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:18 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:18.033+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:18.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:18 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:18.033+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:18.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:18 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:18.033+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:18.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:18 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:18.033+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:18.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:18 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:18.033+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:18.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:18 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:18.033+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.913269+0000 front 2024-03-24T03:12:54.913360+0000 (oldest deadline 2024-03-24T03:13:16.613232+0000) 2024-03-24T03:13:18.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:18 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:18.486+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:18.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:18 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:18.486+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:18.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:18 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:18.486+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:18.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:18 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:18.486+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:18.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:18 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:18.486+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:43.935289+0000 front 2024-03-24T03:12:43.935404+0000 (oldest deadline 2024-03-24T03:13:09.235228+0000) 2024-03-24T03:13:19.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:18 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:18.879+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:19.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:18 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:18.879+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:19.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:18 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:18.879+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:19.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:18 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:18.879+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:19.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:18 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:18.879+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:19.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:19 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:19.014+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:19.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:19 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:19.014+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:19.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:19 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:19.014+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:19.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:19 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:19.014+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:19.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:19 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:19.014+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:19.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:19 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:19.014+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:19.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:19 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:19.014+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.913269+0000 front 2024-03-24T03:12:54.913360+0000 (oldest deadline 2024-03-24T03:13:16.613232+0000) 2024-03-24T03:13:19.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:19 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:19.451+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:15.231716+0000 front 2024-03-24T03:12:15.231495+0000 (oldest deadline 2024-03-24T03:12:36.931574+0000) 2024-03-24T03:13:19.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:19 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:19.451+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:21.032864+0000 front 2024-03-24T03:12:21.032603+0000 (oldest deadline 2024-03-24T03:12:44.532502+0000) 2024-03-24T03:13:19.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:19 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:19.451+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:29.833480+0000 front 2024-03-24T03:12:29.833591+0000 (oldest deadline 2024-03-24T03:12:55.133576+0000) 2024-03-24T03:13:19.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:19 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:19.451+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.133997+0000 front 2024-03-24T03:12:35.134043+0000 (oldest deadline 2024-03-24T03:13:01.034193+0000) 2024-03-24T03:13:19.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:19 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9[66949]: 2024-03-24T03:13:19.451+0000 7f78d233a700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:43.935289+0000 front 2024-03-24T03:12:43.935404+0000 (oldest deadline 2024-03-24T03:13:09.235228+0000) 2024-03-24T03:13:20.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:19 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:19.864+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:20.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:19 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:19.864+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:20.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:19 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:19.864+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:20.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:19 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:19.864+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:20.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:19 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:19.864+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:20.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.001+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:20.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.001+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:20.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.001+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:20.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.001+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:20.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.001+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:20.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.001+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:20.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.001+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.913269+0000 front 2024-03-24T03:12:54.913360+0000 (oldest deadline 2024-03-24T03:13:16.613232+0000) 2024-03-24T03:13:20.706 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:20 smithi017 podman[93094]: 2024-03-24 03:13:20.426804722 +0000 UTC m=+5.437637831 container died d27a961606dfaa01b3112a421e9ffeac5b6ba16e08d261b7c555f072552dbd8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-24T03:13:21.077 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:20 smithi017 podman[93094]: 2024-03-24 03:13:20.827372322 +0000 UTC m=+5.838205440 container cleanup d27a961606dfaa01b3112a421e9ffeac5b6ba16e08d261b7c555f072552dbd8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-24T03:13:21.077 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:20 smithi017 bash[93094]: ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9 2024-03-24T03:13:21.297 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:20 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:20.822+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:21.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:20 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:20.822+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:21.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:20 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:20.822+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:21.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:20 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:20.822+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:21.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:20 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:20.822+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:21.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:20 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:20.822+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:21.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.989+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:21.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.989+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:21.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.989+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:21.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.989+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:21.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.989+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:21.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.989+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:21.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:20 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:20.989+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.913269+0000 front 2024-03-24T03:12:54.913360+0000 (oldest deadline 2024-03-24T03:13:16.613232+0000) 2024-03-24T03:13:21.386 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:21 smithi017 podman[93112]: 2024-03-24 03:13:21.078516808 +0000 UTC m=+0.641405407 container remove d27a961606dfaa01b3112a421e9ffeac5b6ba16e08d261b7c555f072552dbd8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-24T03:13:21.688 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:21 smithi017 podman[93202]: 2024-03-24 03:13:21.387697068 +0000 UTC m=+0.114805406 container create 8d0e39641cccf54ca2f733d42a6a3af734ade267a59b622b0b11d79689c4c9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:13:21.688 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:21 smithi017 podman[93202]: 2024-03-24 03:13:21.290613553 +0000 UTC m=+0.017721857 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T03:13:21.688 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:21 smithi017 podman[93202]: 2024-03-24 03:13:21.606856282 +0000 UTC m=+0.333964578 container init 8d0e39641cccf54ca2f733d42a6a3af734ade267a59b622b0b11d79689c4c9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-24T03:13:21.689 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:21 smithi017 podman[93202]: 2024-03-24 03:13:21.610239977 +0000 UTC m=+0.337348271 container start 8d0e39641cccf54ca2f733d42a6a3af734ade267a59b622b0b11d79689c4c9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9-deactivate, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-24T03:13:21.689 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:21 smithi017 podman[93202]: 2024-03-24 03:13:21.689755176 +0000 UTC m=+0.416863469 container attach 8d0e39641cccf54ca2f733d42a6a3af734ade267a59b622b0b11d79689c4c9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-24T03:13:21.956 INFO:journalctl@ceph.osd.9.smithi017.stdout:Mar 24 03:13:21 smithi017 podman[93202]: 2024-03-24 03:13:21.903336865 +0000 UTC m=+0.630445150 container died 8d0e39641cccf54ca2f733d42a6a3af734ade267a59b622b0b11d79689c4c9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T03:13:22.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:21 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:21.805+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:22.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:21 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:21.805+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:22.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:21 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:21.805+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:22.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:21 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:21.805+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:22.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:21 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:21.805+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:22.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:21 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:21.805+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:22.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:22 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:22.021+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:22.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:22 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:22.021+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:22.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:22 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:22.021+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:22.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:22 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:22.021+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:22.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:22 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:22.021+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:22.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:22 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:22.021+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:22.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:22 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:22.021+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.913269+0000 front 2024-03-24T03:12:54.913360+0000 (oldest deadline 2024-03-24T03:13:16.613232+0000) 2024-03-24T03:13:22.607 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.9.service' 2024-03-24T03:13:22.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:13:22.690 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-24T03:13:22.690 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-24T03:13:22.690 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.10 2024-03-24T03:13:22.820 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:22 smithi019 systemd[1]: Stopping Ceph osd.10 for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:13:23.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:22 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:22.774+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:23.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:22 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:22.774+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:23.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:22 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:22.774+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:23.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:22 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:22.774+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:23.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:22 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:22.774+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:23.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:22 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:22.774+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:23.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:23 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.005+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:23.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:23 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.005+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:23.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:23 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.005+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:23.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:23 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.005+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:23.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:23 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.005+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:23.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:23 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.005+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:23.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:23 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.005+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.913269+0000 front 2024-03-24T03:12:54.913360+0000 (oldest deadline 2024-03-24T03:13:16.613232+0000) 2024-03-24T03:13:23.322 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:23 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.006+0000 7f7beeda5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:13:23.322 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:23 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.006+0000 7f7beeda5700 -1 osd.10 107 *** Got signal Terminated *** 2024-03-24T03:13:23.322 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:23 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.006+0000 7f7beeda5700 -1 osd.10 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T03:13:24.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:23 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:23.811+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:24.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:23 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:23.811+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:24.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:23 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:23.811+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:24.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:23 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:23.811+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:24.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:23 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:23.811+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:24.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:23 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:23.811+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:24.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.999+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:24.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.999+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:24.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.999+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:24.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.999+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:24.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.999+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:24.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.999+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:24.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:23.999+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.913269+0000 front 2024-03-24T03:12:54.913360+0000 (oldest deadline 2024-03-24T03:13:16.613232+0000) 2024-03-24T03:13:25.199 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:24 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:24.851+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:25.200 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:24 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:24.851+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:25.200 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:24 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:24.851+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:25.200 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:24 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:24.851+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:25.200 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:24 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:24.851+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:25.200 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:24 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:24.851+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:25.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:24.951+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:25.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:24.951+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:25.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:24.951+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:25.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:24.951+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:25.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:24.951+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:25.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:24.951+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:25.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:24 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:24.951+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.913269+0000 front 2024-03-24T03:12:54.913360+0000 (oldest deadline 2024-03-24T03:13:16.613232+0000) 2024-03-24T03:13:26.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:25 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:25.831+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:26.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:25 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:25.831+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:26.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:25 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:25.831+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:26.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:25 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:25.831+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:26.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:25 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:25.831+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:26.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:25 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:25.831+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:26.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:25 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:25.983+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:26.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:25 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:25.983+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:26.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:25 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:25.983+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:26.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:25 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:25.983+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:26.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:25 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:25.983+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:26.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:25 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:25.983+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:26.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:25 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:25.983+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.913269+0000 front 2024-03-24T03:12:54.913360+0000 (oldest deadline 2024-03-24T03:13:16.613232+0000) 2024-03-24T03:13:27.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:26 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:26.806+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:27.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:26 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:26.806+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:27.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:26 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:26.806+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:27.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:26 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:26.806+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:27.298 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:26 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:26.806+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:27.299 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:26 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:26.806+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:27.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:27 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:27.028+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:07.006886+0000 front 2024-03-24T03:12:07.006728+0000 (oldest deadline 2024-03-24T03:12:32.306840+0000) 2024-03-24T03:13:27.320 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:27 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:27.028+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:12.307215+0000 front 2024-03-24T03:12:12.307568+0000 (oldest deadline 2024-03-24T03:12:37.607350+0000) 2024-03-24T03:13:27.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:27 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:27.028+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.308451+0000 front 2024-03-24T03:12:22.308384+0000 (oldest deadline 2024-03-24T03:12:46.408434+0000) 2024-03-24T03:13:27.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:27 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:27.028+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-03-24T03:12:31.709518+0000 front 2024-03-24T03:12:31.709621+0000 (oldest deadline 2024-03-24T03:12:52.209435+0000) 2024-03-24T03:13:27.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:27 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:27.028+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:35.110442+0000 front 2024-03-24T03:12:35.110388+0000 (oldest deadline 2024-03-24T03:13:01.510928+0000) 2024-03-24T03:13:27.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:27 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:27.028+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:45.611823+0000 front 2024-03-24T03:12:45.611925+0000 (oldest deadline 2024-03-24T03:13:09.711762+0000) 2024-03-24T03:13:27.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:27 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:27.028+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.913269+0000 front 2024-03-24T03:12:54.913360+0000 (oldest deadline 2024-03-24T03:13:16.613232+0000) 2024-03-24T03:13:27.321 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:27 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10[60816]: 2024-03-24T03:13:27.028+0000 7f7beb399700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-03-24T03:13:04.214690+0000 front 2024-03-24T03:13:04.214791+0000 (oldest deadline 2024-03-24T03:13:26.514703+0000) 2024-03-24T03:13:28.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:27 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:27.765+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:28.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:27 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:27.765+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:28.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:27 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:27.765+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:28.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:27 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:27.765+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:28.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:27 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:27.765+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:28.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:27 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:27.765+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:28.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:27 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:27.765+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-03-24T03:13:04.190593+0000 front 2024-03-24T03:13:04.190606+0000 (oldest deadline 2024-03-24T03:13:27.090618+0000) 2024-03-24T03:13:28.589 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:28 smithi019 podman[72855]: 2024-03-24 03:13:28.340049891 +0000 UTC m=+5.456105396 container died 7222f8a8ab5c3bece3edb6aa9dce91d407fddb2dc22c2941c34ff6a52603c7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:13:28.840 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:28 smithi019 podman[72855]: 2024-03-24 03:13:28.673346629 +0000 UTC m=+5.789402104 container cleanup 7222f8a8ab5c3bece3edb6aa9dce91d407fddb2dc22c2941c34ff6a52603c7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True) 2024-03-24T03:13:28.840 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:28 smithi019 bash[72855]: ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10 2024-03-24T03:13:29.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:28 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:28.780+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:29.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:28 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:28.780+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:29.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:28 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:28.780+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:29.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:28 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:28.780+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:29.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:28 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:28.780+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:29.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:28 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:28.780+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:29.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:28 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:28.780+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-03-24T03:13:04.190593+0000 front 2024-03-24T03:13:04.190606+0000 (oldest deadline 2024-03-24T03:13:27.090618+0000) 2024-03-24T03:13:29.248 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:28 smithi019 podman[72872]: 2024-03-24 03:13:28.841097159 +0000 UTC m=+0.494337246 container remove 7222f8a8ab5c3bece3edb6aa9dce91d407fddb2dc22c2941c34ff6a52603c7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T03:13:29.527 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:29 smithi019 podman[72963]: 2024-03-24 03:13:29.152752084 +0000 UTC m=+0.018968524 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T03:13:29.527 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:29 smithi019 podman[72963]: 2024-03-24 03:13:29.250520528 +0000 UTC m=+0.116737011 container create 7f15ab65c84a66431a7e79bc53d6c334e8e70dcc9ce2a4959de55ed433364f52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319) 2024-03-24T03:13:29.527 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:29 smithi019 podman[72963]: 2024-03-24 03:13:29.44454708 +0000 UTC m=+0.310763532 container init 7f15ab65c84a66431a7e79bc53d6c334e8e70dcc9ce2a4959de55ed433364f52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:13:29.527 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:29 smithi019 podman[72963]: 2024-03-24 03:13:29.447828877 +0000 UTC m=+0.314045318 container start 7f15ab65c84a66431a7e79bc53d6c334e8e70dcc9ce2a4959de55ed433364f52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T03:13:29.820 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:29 smithi019 podman[72963]: 2024-03-24 03:13:29.527723629 +0000 UTC m=+0.393940079 container attach 7f15ab65c84a66431a7e79bc53d6c334e8e70dcc9ce2a4959de55ed433364f52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-24T03:13:29.821 INFO:journalctl@ceph.osd.10.smithi019.stdout:Mar 24 03:13:29 smithi019 podman[72963]: 2024-03-24 03:13:29.745290495 +0000 UTC m=+0.611506943 container died 7f15ab65c84a66431a7e79bc53d6c334e8e70dcc9ce2a4959de55ed433364f52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-03-24T03:13:30.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:29 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:29.762+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:30.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:29 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:29.762+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:30.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:29 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:29.762+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:30.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:29 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:29.762+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:30.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:29 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:29.762+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:30.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:29 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:29.762+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:30.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:29 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:29.762+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-03-24T03:13:04.190593+0000 front 2024-03-24T03:13:04.190606+0000 (oldest deadline 2024-03-24T03:13:27.090618+0000) 2024-03-24T03:13:30.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.10.service' 2024-03-24T03:13:30.380 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:13:30.381 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-24T03:13:30.381 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-24T03:13:30.381 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.11 2024-03-24T03:13:30.674 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:30 smithi188 systemd[1]: Stopping Ceph osd.11 for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:13:31.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:30 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:30.674+0000 7faeac751700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:13:31.049 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:30 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:30.674+0000 7faeac751700 -1 osd.11 107 *** Got signal Terminated *** 2024-03-24T03:13:31.049 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:30 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:30.674+0000 7faeac751700 -1 osd.11 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T03:13:31.049 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:30 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:30.770+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:31.049 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:30 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:30.770+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:31.049 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:30 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:30.770+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:31.049 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:30 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:30.770+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:31.049 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:30 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:30.770+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:31.049 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:30 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:30.770+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:31.049 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:30 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:30.770+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-03-24T03:13:04.190593+0000 front 2024-03-24T03:13:04.190606+0000 (oldest deadline 2024-03-24T03:13:27.090618+0000) 2024-03-24T03:13:32.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:31 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:31.742+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:32.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:31 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:31.742+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:32.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:31 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:31.742+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:32.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:31 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:31.742+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:32.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:31 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:31.742+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:32.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:31 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:31.742+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:32.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:31 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:31.742+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-03-24T03:13:04.190593+0000 front 2024-03-24T03:13:04.190606+0000 (oldest deadline 2024-03-24T03:13:27.090618+0000) 2024-03-24T03:13:33.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:32 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:32.747+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:33.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:32 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:32.747+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:33.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:32 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:32.747+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:33.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:32 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:32.747+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:33.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:32 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:32.747+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:33.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:32 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:32.747+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:33.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:32 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:32.747+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-03-24T03:13:04.190593+0000 front 2024-03-24T03:13:04.190606+0000 (oldest deadline 2024-03-24T03:13:27.090618+0000) 2024-03-24T03:13:34.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:33 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:33.764+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:34.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:33 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:33.764+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:34.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:33 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:33.764+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:34.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:33 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:33.764+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:34.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:33 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:33.764+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:34.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:33 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:33.764+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:34.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:33 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:33.764+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-03-24T03:13:04.190593+0000 front 2024-03-24T03:13:04.190606+0000 (oldest deadline 2024-03-24T03:13:27.090618+0000) 2024-03-24T03:13:34.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:33 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:33.764+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6820 osd.8 since back 2024-03-24T03:13:07.091096+0000 front 2024-03-24T03:13:07.091121+0000 (oldest deadline 2024-03-24T03:13:32.991022+0000) 2024-03-24T03:13:35.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:34 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:34.745+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-03-24T03:12:04.980733+0000 front 2024-03-24T03:12:04.980896+0000 (oldest deadline 2024-03-24T03:12:28.480836+0000) 2024-03-24T03:13:35.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:34 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:34.745+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6804 osd.1 since back 2024-03-24T03:12:14.183383+0000 front 2024-03-24T03:12:14.183344+0000 (oldest deadline 2024-03-24T03:12:37.083172+0000) 2024-03-24T03:13:35.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:34 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:34.745+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-03-24T03:12:22.384361+0000 front 2024-03-24T03:12:22.384346+0000 (oldest deadline 2024-03-24T03:12:45.284373+0000) 2024-03-24T03:13:35.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:34 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:34.745+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6812 osd.4 since back 2024-03-24T03:12:37.486598+0000 front 2024-03-24T03:12:37.486561+0000 (oldest deadline 2024-03-24T03:13:01.586592+0000) 2024-03-24T03:13:35.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:34 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:34.745+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-03-24T03:12:47.387997+0000 front 2024-03-24T03:12:47.388083+0000 (oldest deadline 2024-03-24T03:13:08.488121+0000) 2024-03-24T03:13:35.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:34 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:34.745+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-03-24T03:12:54.889568+0000 front 2024-03-24T03:12:54.889535+0000 (oldest deadline 2024-03-24T03:13:20.189339+0000) 2024-03-24T03:13:35.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:34 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:34.745+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.19:6820 osd.7 since back 2024-03-24T03:13:04.190593+0000 front 2024-03-24T03:13:04.190606+0000 (oldest deadline 2024-03-24T03:13:27.090618+0000) 2024-03-24T03:13:35.048 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:34 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11[61685]: 2024-03-24T03:13:34.745+0000 7faea4d3d700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.188:6820 osd.8 since back 2024-03-24T03:13:07.091096+0000 front 2024-03-24T03:13:07.091121+0000 (oldest deadline 2024-03-24T03:13:32.991022+0000) 2024-03-24T03:13:36.332 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:36 smithi188 podman[73445]: 2024-03-24 03:13:36.081312811 +0000 UTC m=+5.506745257 container died 5543f5d3374b9fb9a447c9a09fe655375edc78a0bd9a80a7217a5b63c36e4f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-03-24T03:13:36.590 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:36 smithi188 podman[73445]: 2024-03-24 03:13:36.424041403 +0000 UTC m=+5.849473846 container cleanup 5543f5d3374b9fb9a447c9a09fe655375edc78a0bd9a80a7217a5b63c36e4f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T03:13:36.591 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:36 smithi188 bash[73445]: ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11 2024-03-24T03:13:36.998 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:36 smithi188 podman[73463]: 2024-03-24 03:13:36.591793989 +0000 UTC m=+0.510336138 container remove 5543f5d3374b9fb9a447c9a09fe655375edc78a0bd9a80a7217a5b63c36e4f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T03:13:37.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:36 smithi188 podman[73551]: 2024-03-24 03:13:36.902110244 +0000 UTC m=+0.017558015 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T03:13:37.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:37 smithi188 podman[73551]: 2024-03-24 03:13:37.000751259 +0000 UTC m=+0.116199043 container create f1d45e7330e074516779c799fa04e2b6cc2d129384354220d1a172a57164d5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-24T03:13:37.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:37 smithi188 podman[73551]: 2024-03-24 03:13:37.1950635 +0000 UTC m=+0.310511319 container init f1d45e7330e074516779c799fa04e2b6cc2d129384354220d1a172a57164d5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-24T03:13:37.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:37 smithi188 podman[73551]: 2024-03-24 03:13:37.198482338 +0000 UTC m=+0.313930112 container start f1d45e7330e074516779c799fa04e2b6cc2d129384354220d1a172a57164d5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-24T03:13:37.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:37 smithi188 podman[73551]: 2024-03-24 03:13:37.269617923 +0000 UTC m=+0.385065694 container attach f1d45e7330e074516779c799fa04e2b6cc2d129384354220d1a172a57164d5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-24T03:13:37.548 INFO:journalctl@ceph.osd.11.smithi188.stdout:Mar 24 03:13:37 smithi188 podman[73551]: 2024-03-24 03:13:37.478671764 +0000 UTC m=+0.594119535 container died f1d45e7330e074516779c799fa04e2b6cc2d129384354220d1a172a57164d5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-24T03:13:38.146 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@osd.11.service' 2024-03-24T03:13:38.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:13:38.189 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-24T03:13:38.189 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-24T03:13:38.189 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.a 2024-03-24T03:13:38.496 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 03:13:38 smithi017 systemd[1]: Stopping Ceph mds.a for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:13:38.763 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 03:13:38 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-a[73909]: 2024-03-24T03:13:38.495+0000 7f5f77d22700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:13:38.763 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 03:13:38 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-a[73909]: 2024-03-24T03:13:38.495+0000 7f5f77d22700 -1 mds.a *** got signal Terminated *** 2024-03-24T03:13:42.629 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 03:13:42 smithi017 podman[93319]: 2024-03-24 03:13:42.227963206 +0000 UTC m=+3.836990477 container died 0b11fbe4e1a7539b55bb4ce9f378f17f7ef224783043fd40b2e887af8d040750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.build-date=20240319) 2024-03-24T03:13:42.956 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 03:13:42 smithi017 podman[93319]: 2024-03-24 03:13:42.728788171 +0000 UTC m=+4.337815452 container cleanup 0b11fbe4e1a7539b55bb4ce9f378f17f7ef224783043fd40b2e887af8d040750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-24T03:13:42.956 INFO:journalctl@ceph.mds.a.smithi017.stdout:Mar 24 03:13:42 smithi017 bash[93319]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-a 2024-03-24T03:13:43.077 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.a.service' 2024-03-24T03:13:43.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:13:43.115 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-24T03:13:43.115 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-24T03:13:43.115 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.d 2024-03-24T03:13:43.440 INFO:journalctl@ceph.mds.d.smithi017.stdout:Mar 24 03:13:43 smithi017 systemd[1]: Stopping Ceph mds.d for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:13:43.706 INFO:journalctl@ceph.mds.d.smithi017.stdout:Mar 24 03:13:43 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-d[72945]: 2024-03-24T03:13:43.440+0000 7f64405c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:13:43.706 INFO:journalctl@ceph.mds.d.smithi017.stdout:Mar 24 03:13:43 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-d[72945]: 2024-03-24T03:13:43.440+0000 7f64405c5700 -1 mds.d *** got signal Terminated *** 2024-03-24T03:13:47.706 INFO:journalctl@ceph.mds.d.smithi017.stdout:Mar 24 03:13:47 smithi017 podman[93452]: 2024-03-24 03:13:47.4496505 +0000 UTC m=+4.115807716 container died 7a5f191e5486a40c8353ef1f151c259abeaf401a3553577cdc1743ba64ab8c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-24T03:13:48.091 INFO:journalctl@ceph.mds.d.smithi017.stdout:Mar 24 03:13:47 smithi017 podman[93452]: 2024-03-24 03:13:47.807354061 +0000 UTC m=+4.473511279 container cleanup 7a5f191e5486a40c8353ef1f151c259abeaf401a3553577cdc1743ba64ab8c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-24T03:13:48.091 INFO:journalctl@ceph.mds.d.smithi017.stdout:Mar 24 03:13:47 smithi017 bash[93452]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-d 2024-03-24T03:13:48.232 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.d.service' 2024-03-24T03:13:48.267 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:13:48.268 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-24T03:13:48.268 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-24T03:13:48.268 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.g 2024-03-24T03:13:48.706 INFO:journalctl@ceph.mds.g.smithi017.stdout:Mar 24 03:13:48 smithi017 systemd[1]: Stopping Ceph mds.g for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:13:48.706 INFO:journalctl@ceph.mds.g.smithi017.stdout:Mar 24 03:13:48 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-g[73549]: 2024-03-24T03:13:48.585+0000 7ffbe8ccc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:13:48.706 INFO:journalctl@ceph.mds.g.smithi017.stdout:Mar 24 03:13:48 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-g[73549]: 2024-03-24T03:13:48.585+0000 7ffbe8ccc700 -1 mds.g *** got signal Terminated *** 2024-03-24T03:13:50.478 INFO:journalctl@ceph.mds.g.smithi017.stdout:Mar 24 03:13:50 smithi017 podman[93573]: 2024-03-24 03:13:50.214844576 +0000 UTC m=+1.733442312 container died e3ab97c6dd883b7b6e4f0079fa6c760fefdd16c0a7fe67186b2877a13acb3584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-g, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-24T03:13:50.836 INFO:journalctl@ceph.mds.g.smithi017.stdout:Mar 24 03:13:50 smithi017 podman[93573]: 2024-03-24 03:13:50.586489105 +0000 UTC m=+2.105086849 container cleanup e3ab97c6dd883b7b6e4f0079fa6c760fefdd16c0a7fe67186b2877a13acb3584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-g, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-24T03:13:50.836 INFO:journalctl@ceph.mds.g.smithi017.stdout:Mar 24 03:13:50 smithi017 bash[93573]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-g 2024-03-24T03:13:50.969 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.g.service' 2024-03-24T03:13:51.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:13:51.004 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-24T03:13:51.004 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-24T03:13:51.004 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.j 2024-03-24T03:13:51.456 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 03:13:51 smithi017 systemd[1]: Stopping Ceph mds.j for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:13:51.456 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 03:13:51 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-j[72593]: 2024-03-24T03:13:51.330+0000 7f6e11a14700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:13:51.456 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 03:13:51 smithi017 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-j[72593]: 2024-03-24T03:13:51.330+0000 7f6e11a14700 -1 mds.j *** got signal Terminated *** 2024-03-24T03:13:53.449 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 03:13:53 smithi017 podman[93695]: 2024-03-24 03:13:53.146019698 +0000 UTC m=+1.918734956 container died 37509107601a2dc6fc2c58c43304467276652060dae8f203bef6cdb45b780904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-j, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-24T03:13:53.706 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 03:13:53 smithi017 podman[93695]: 2024-03-24 03:13:53.531828123 +0000 UTC m=+2.304543389 container cleanup 37509107601a2dc6fc2c58c43304467276652060dae8f203bef6cdb45b780904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-j, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-24T03:13:53.706 INFO:journalctl@ceph.mds.j.smithi017.stdout:Mar 24 03:13:53 smithi017 bash[93695]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-j 2024-03-24T03:13:53.903 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.j.service' 2024-03-24T03:13:53.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:13:53.936 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-24T03:13:53.936 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-24T03:13:53.936 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.b 2024-03-24T03:13:54.274 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 03:13:53 smithi019 systemd[1]: Stopping Ceph mds.b for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:13:54.552 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 03:13:54 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b[66492]: 2024-03-24T03:13:54.274+0000 7f68b3c2a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:13:54.553 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 03:13:54 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b[66492]: 2024-03-24T03:13:54.274+0000 7f68b3c2a700 -1 mds.b *** got signal Terminated *** 2024-03-24T03:13:58.564 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 03:13:58 smithi019 podman[73069]: 2024-03-24 03:13:58.229245054 +0000 UTC m=+4.086363052 container died 0c7ee800196e6ff2fdfba445d337b3f7d4c03f7e5926ce9a89641b4ed4af4c79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-24T03:13:58.564 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 03:13:58 smithi019 podman[73069]: 2024-03-24 03:13:58.56545532 +0000 UTC m=+4.422573311 container cleanup 0c7ee800196e6ff2fdfba445d337b3f7d4c03f7e5926ce9a89641b4ed4af4c79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T03:13:58.820 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 03:13:58 smithi019 bash[73069]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b 2024-03-24T03:13:58.820 INFO:journalctl@ceph.mds.b.smithi019.stdout:Mar 24 03:13:58 smithi019 podman[73096]: 2024-03-24 03:13:58.783304861 +0000 UTC m=+0.549792236 container remove 0c7ee800196e6ff2fdfba445d337b3f7d4c03f7e5926ce9a89641b4ed4af4c79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-b, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-24T03:13:58.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.b.service' 2024-03-24T03:13:59.035 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:13:59.036 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-24T03:13:59.036 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-24T03:13:59.036 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.e 2024-03-24T03:13:59.321 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 03:13:59 smithi019 systemd[1]: Stopping Ceph mds.e for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:13:59.820 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 03:13:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e[65869]: 2024-03-24T03:13:59.377+0000 7febc84f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:13:59.821 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 03:13:59 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e[65869]: 2024-03-24T03:13:59.377+0000 7febc84f8700 -1 mds.e *** got signal Terminated *** 2024-03-24T03:14:02.570 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 03:14:02 smithi019 podman[73203]: 2024-03-24 03:14:02.221269224 +0000 UTC m=+2.953942377 container died 2d477d3fb304776e074101b02ab6e3dad0277da7fb182e8798b21d6f714e55c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:14:03.033 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 03:14:02 smithi019 podman[73203]: 2024-03-24 03:14:02.661233106 +0000 UTC m=+3.393906255 container cleanup 2d477d3fb304776e074101b02ab6e3dad0277da7fb182e8798b21d6f714e55c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T03:14:03.033 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 03:14:02 smithi019 bash[73203]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e 2024-03-24T03:14:03.033 INFO:journalctl@ceph.mds.e.smithi019.stdout:Mar 24 03:14:02 smithi019 podman[73217]: 2024-03-24 03:14:02.829494394 +0000 UTC m=+0.611191862 container remove 2d477d3fb304776e074101b02ab6e3dad0277da7fb182e8798b21d6f714e55c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-24T03:14:03.043 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.e.service' 2024-03-24T03:14:03.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:14:03.083 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-24T03:14:03.083 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-24T03:14:03.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.h 2024-03-24T03:14:03.320 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 03:14:03 smithi019 systemd[1]: Stopping Ceph mds.h for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:14:03.820 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 03:14:03 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h[66184]: 2024-03-24T03:14:03.406+0000 7f205f61e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:14:03.821 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 03:14:03 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h[66184]: 2024-03-24T03:14:03.406+0000 7f205f61e700 -1 mds.h *** got signal Terminated *** 2024-03-24T03:14:06.505 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 03:14:06 smithi019 podman[73322]: 2024-03-24 03:14:06.229250041 +0000 UTC m=+2.920518251 container died 35e3e6dbc206d3a6b4a492926ba08d1b90c25e6c800ca5367e03b7a04683f188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-24T03:14:06.506 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 03:14:06 smithi019 podman[73322]: 2024-03-24 03:14:06.506744168 +0000 UTC m=+3.198012385 container cleanup 35e3e6dbc206d3a6b4a492926ba08d1b90c25e6c800ca5367e03b7a04683f188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T03:14:06.820 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 03:14:06 smithi019 bash[73322]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h 2024-03-24T03:14:06.820 INFO:journalctl@ceph.mds.h.smithi019.stdout:Mar 24 03:14:06 smithi019 podman[73336]: 2024-03-24 03:14:06.749704732 +0000 UTC m=+0.518142682 container remove 35e3e6dbc206d3a6b4a492926ba08d1b90c25e6c800ca5367e03b7a04683f188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T03:14:06.888 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.h.service' 2024-03-24T03:14:06.926 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:14:06.926 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-24T03:14:06.926 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-24T03:14:06.926 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.k 2024-03-24T03:14:07.259 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 03:14:07 smithi019 systemd[1]: Stopping Ceph mds.k for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:14:07.570 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 03:14:07 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k[65548]: 2024-03-24T03:14:07.259+0000 7f6b555e9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:14:07.570 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 03:14:07 smithi019 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k[65548]: 2024-03-24T03:14:07.259+0000 7f6b555e9700 -1 mds.k *** got signal Terminated *** 2024-03-24T03:14:10.518 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 03:14:10 smithi019 podman[73445]: 2024-03-24 03:14:10.237390972 +0000 UTC m=+3.086775122 container died 535b6a63956fdf12270ae3946bb95a03ac362f58f54ea2f70a9081653fc18091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0) 2024-03-24T03:14:10.518 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 03:14:10 smithi019 podman[73445]: 2024-03-24 03:14:10.519414 +0000 UTC m=+3.368798151 container cleanup 535b6a63956fdf12270ae3946bb95a03ac362f58f54ea2f70a9081653fc18091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True) 2024-03-24T03:14:10.820 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 03:14:10 smithi019 bash[73445]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k 2024-03-24T03:14:10.820 INFO:journalctl@ceph.mds.k.smithi019.stdout:Mar 24 03:14:10 smithi019 podman[73459]: 2024-03-24 03:14:10.671694123 +0000 UTC m=+0.431723704 container remove 535b6a63956fdf12270ae3946bb95a03ac362f58f54ea2f70a9081653fc18091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-k, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:14:10.890 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.k.service' 2024-03-24T03:14:10.927 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:14:10.928 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-24T03:14:10.928 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-24T03:14:10.928 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.c 2024-03-24T03:14:11.278 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 03:14:10 smithi188 systemd[1]: Stopping Ceph mds.c for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:14:11.548 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 03:14:11 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-c[66846]: 2024-03-24T03:14:11.335+0000 7fe82d4d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:14:11.548 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 03:14:11 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-c[66846]: 2024-03-24T03:14:11.335+0000 7fe82d4d3700 -1 mds.c *** got signal Terminated *** 2024-03-24T03:14:14.548 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 03:14:14 smithi188 podman[73656]: 2024-03-24 03:14:14.23002119 +0000 UTC m=+3.075966411 container died aab8aa032050a71cc7afb5bd6867ffe5defda82cb11ed42b2c6445b87997da1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5) 2024-03-24T03:14:15.048 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 03:14:14 smithi188 podman[73656]: 2024-03-24 03:14:14.777147774 +0000 UTC m=+3.623092985 container cleanup aab8aa032050a71cc7afb5bd6867ffe5defda82cb11ed42b2c6445b87997da1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-24T03:14:15.048 INFO:journalctl@ceph.mds.c.smithi188.stdout:Mar 24 03:14:14 smithi188 bash[73656]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-c 2024-03-24T03:14:15.194 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.c.service' 2024-03-24T03:14:15.233 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:14:15.233 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-24T03:14:15.233 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-24T03:14:15.233 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.f 2024-03-24T03:14:15.798 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 03:14:15 smithi188 systemd[1]: Stopping Ceph mds.f for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:14:15.798 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 03:14:15 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-f[66535]: 2024-03-24T03:14:15.546+0000 7f71da26f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:14:15.798 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 03:14:15 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-f[66535]: 2024-03-24T03:14:15.546+0000 7f71da26f700 -1 mds.f *** got signal Terminated *** 2024-03-24T03:14:18.496 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 03:14:18 smithi188 podman[73786]: 2024-03-24 03:14:18.230431361 +0000 UTC m=+2.783413133 container died 8d20f8a515d950eb5b34e75aa2e6fcf08d76972aa7b74283500edddc7a567eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-24T03:14:18.798 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 03:14:18 smithi188 podman[73786]: 2024-03-24 03:14:18.599477331 +0000 UTC m=+3.152459111 container cleanup 8d20f8a515d950eb5b34e75aa2e6fcf08d76972aa7b74283500edddc7a567eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-24T03:14:18.798 INFO:journalctl@ceph.mds.f.smithi188.stdout:Mar 24 03:14:18 smithi188 bash[73786]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-f 2024-03-24T03:14:18.970 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.f.service' 2024-03-24T03:14:19.005 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:14:19.006 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-24T03:14:19.006 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-24T03:14:19.006 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.i 2024-03-24T03:14:19.548 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 03:14:19 smithi188 systemd[1]: Stopping Ceph mds.i for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:14:19.548 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 03:14:19 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i[67156]: 2024-03-24T03:14:19.327+0000 7f7738a23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:14:19.548 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 03:14:19 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i[67156]: 2024-03-24T03:14:19.327+0000 7f7738a23700 -1 mds.i *** got signal Terminated *** 2024-03-24T03:14:22.494 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 03:14:22 smithi188 podman[73906]: 2024-03-24 03:14:22.226823151 +0000 UTC m=+3.003956246 container died f3ce1e99fefa3041537e9646cb74de250e2b728a3543acb85bd79f7891d13bbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-24T03:14:22.798 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 03:14:22 smithi188 podman[73906]: 2024-03-24 03:14:22.586315122 +0000 UTC m=+3.363448210 container cleanup f3ce1e99fefa3041537e9646cb74de250e2b728a3543acb85bd79f7891d13bbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T03:14:22.798 INFO:journalctl@ceph.mds.i.smithi188.stdout:Mar 24 03:14:22 smithi188 bash[73906]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-i 2024-03-24T03:14:23.001 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.i.service' 2024-03-24T03:14:23.035 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:14:23.035 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-24T03:14:23.035 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-24T03:14:23.035 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.l 2024-03-24T03:14:23.548 INFO:journalctl@ceph.mds.l.smithi188.stdout:Mar 24 03:14:23 smithi188 systemd[1]: Stopping Ceph mds.l for 2a682800-e989-11ee-95cd-87774f69a715... 2024-03-24T03:14:23.548 INFO:journalctl@ceph.mds.l.smithi188.stdout:Mar 24 03:14:23 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-l[66224]: 2024-03-24T03:14:23.347+0000 7f0974ce0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T03:14:23.548 INFO:journalctl@ceph.mds.l.smithi188.stdout:Mar 24 03:14:23 smithi188 ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-l[66224]: 2024-03-24T03:14:23.347+0000 7f0974ce0700 -1 mds.l *** got signal Terminated *** 2024-03-24T03:14:26.531 INFO:journalctl@ceph.mds.l.smithi188.stdout:Mar 24 03:14:26 smithi188 podman[74027]: 2024-03-24 03:14:26.218024741 +0000 UTC m=+2.968763937 container died 56139eb4601b85cab59be9796c5a1fd2fe915d8a8c337e4441d37cc01a9c6983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-24T03:14:26.798 INFO:journalctl@ceph.mds.l.smithi188.stdout:Mar 24 03:14:26 smithi188 podman[74027]: 2024-03-24 03:14:26.615132406 +0000 UTC m=+3.365871630 container cleanup 56139eb4601b85cab59be9796c5a1fd2fe915d8a8c337e4441d37cc01a9c6983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-l, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T03:14:26.798 INFO:journalctl@ceph.mds.l.smithi188.stdout:Mar 24 03:14:26 smithi188 bash[74027]: ceph-2a682800-e989-11ee-95cd-87774f69a715-mds-l 2024-03-24T03:14:26.979 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a682800-e989-11ee-95cd-87774f69a715@mds.l.service' 2024-03-24T03:14:27.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T03:14:27.013 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-24T03:14:27.013 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a682800-e989-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-24T03:14:27.210 INFO:teuthology.orchestra.run.smithi017.stdout:Deleting cluster with fsid: 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T03:14:29.677 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a682800-e989-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-24T03:14:29.879 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T03:14:32.501 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a682800-e989-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-24T03:14:32.690 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T03:14:35.151 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T03:14:35.186 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T03:14:35.225 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T03:14:35.253 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-24T03:14:35.254 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480/remote/smithi017/crash 2024-03-24T03:14:35.255 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/crash -- . 2024-03-24T03:14:35.297 INFO:teuthology.orchestra.run.smithi017.stderr:tar: /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-24T03:14:35.297 INFO:teuthology.orchestra.run.smithi017.stderr:tar: Error is not recoverable: exiting now 2024-03-24T03:14:35.299 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480/remote/smithi019/crash 2024-03-24T03:14:35.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/crash -- . 2024-03-24T03:14:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-24T03:14:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-03-24T03:14:35.348 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480/remote/smithi188/crash 2024-03-24T03:14:35.349 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/crash -- . 2024-03-24T03:14:35.394 INFO:teuthology.orchestra.run.smithi188.stderr:tar: /var/lib/ceph/2a682800-e989-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-24T03:14:35.394 INFO:teuthology.orchestra.run.smithi188.stderr:tar: Error is not recoverable: exiting now 2024-03-24T03:14:35.395 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-24T03:14:35.395 DEBUG:teuthology.orchestra.run.smithi017:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-24T03:14:35.432 INFO:teuthology.orchestra.run.smithi017.stdout:2024-03-24T02:52:29.811398+0000 mon.a (mon.0) 189 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-24T03:14:35.433 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-24T03:14:35.433 DEBUG:teuthology.orchestra.run.smithi017:> sudo egrep '\[SEC\]' /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-24T03:14:35.508 DEBUG:teuthology.orchestra.run.smithi017:> sudo egrep '\[ERR\]' /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-24T03:14:35.584 DEBUG:teuthology.orchestra.run.smithi017:> sudo egrep '\[WRN\]' /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-24T03:14:35.659 INFO:teuthology.orchestra.run.smithi017.stdout:2024-03-24T02:52:29.811398+0000 mon.a (mon.0) 189 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-24T03:14:35.659 INFO:tasks.cephadm:Compressing logs... 2024-03-24T03:14:35.659 DEBUG:teuthology.orchestra.run.smithi017:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-24T03:14:35.702 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-24T03:14:35.705 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-24T03:14:35.727 INFO:teuthology.orchestra.run.smithi017.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-24T03:14:35.727 INFO:teuthology.orchestra.run.smithi017.stderr:: No such file or directory 2024-03-24T03:14:35.727 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-24T03:14:35.729 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-24T03:14:35.729 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-24T03:14:35.729 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.g.log 2024-03-24T03:14:35.729 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.j.log 2024-03-24T03:14:35.730 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.audit.log: /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-24T03:14:35.731 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-24T03:14:35.731 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-24T03:14:35.731 INFO:teuthology.orchestra.run.smithi188.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-24T03:14:35.731 INFO:teuthology.orchestra.run.smithi188.stderr:: No such file or directory 2024-03-24T03:14:35.732 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.l.log 2024-03-24T03:14:35.732 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose --/var/log/ceph/cephadm.log: /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-24T03:14:35.732 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.i.log 2024-03-24T03:14:35.732 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.l.log: /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.11.log 2024-03-24T03:14:35.733 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mgr.x.log: 90.3% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-24T03:14:35.737 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mgr.z.log 2024-03-24T03:14:35.737 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.i.log: /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.8.log 2024-03-24T03:14:35.737 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mgr.z.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-24T03:14:35.738 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-24T03:14:35.738 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.f.log 2024-03-24T03:14:35.738 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-24T03:14:35.738 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.f.log: 90.5% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-24T03:14:35.738 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log 2024-03-24T03:14:35.738 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-24T03:14:35.738 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-24T03:14:35.740 INFO:teuthology.orchestra.run.smithi017.stderr: 94.7% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.j.log.gz 2024-03-24T03:14:35.740 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-24T03:14:35.740 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log 2024-03-24T03:14:35.740 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.cephadm.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 84.3% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-24T03:14:35.740 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-24T03:14:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-03-24T03:14:35.742 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-24T03:14:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.10.log 2024-03-24T03:14:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-24T03:14:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.b.log 2024-03-24T03:14:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-24T03:14:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.audit.log: /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-24T03:14:35.742 INFO:teuthology.orchestra.run.smithi019.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-24T03:14:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.k.log 2024-03-24T03:14:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.h.log 2024-03-24T03:14:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-24T03:14:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.e.log 2024-03-24T03:14:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.7.log: 90.4% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-24T03:14:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-24T03:14:35.745 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log: 87.3% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-24T03:14:35.746 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.c.log 2024-03-24T03:14:35.746 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.2.log: 91.8% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mgr.z.log.gz 2024-03-24T03:14:35.746 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-24T03:14:35.751 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-24T03:14:35.752 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.9.log 2024-03-24T03:14:35.752 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-24T03:14:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-24T03:14:35.754 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-volume.log: 87.2% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-24T03:14:35.764 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.d.log 2024-03-24T03:14:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.9.log: 91.0% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-24T03:14:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.a.log 2024-03-24T03:14:35.772 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.d.log.gz 2024-03-24T03:14:35.773 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-24T03:14:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mgr.y.log: 91.8% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-24T03:14:35.774 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log 2024-03-24T03:14:35.774 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-24T03:14:35.780 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mon.c.log: 91.0% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-24T03:14:35.783 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-24T03:14:35.785 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-24T03:14:35.792 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log: 87.3%gzip -5 --verbose -- /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-24T03:14:35.792 INFO:teuthology.orchestra.run.smithi188.stderr: 93.4% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.f.log.gz 2024-03-24T03:14:35.796 INFO:teuthology.orchestra.run.smithi017.stderr: 94.5% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.g.log.gz 2024-03-24T03:14:35.797 INFO:teuthology.orchestra.run.smithi188.stderr: 93.6% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.c.log.gz 2024-03-24T03:14:35.798 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-volume.log: -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-24T03:14:35.799 INFO:teuthology.orchestra.run.smithi019.stderr: 93.6% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.h.log.gz 2024-03-24T03:14:35.799 INFO:teuthology.orchestra.run.smithi019.stderr: 93.5% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.b.log.gz 2024-03-24T03:14:35.810 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mon.b.log: 94.5% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.k.log.gz 2024-03-24T03:14:35.813 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-24T03:14:35.814 INFO:teuthology.orchestra.run.smithi188.stderr: 94.6% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.l.log.gz 2024-03-24T03:14:35.843 INFO:teuthology.orchestra.run.smithi017.stderr: 94.5% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.a.log.gz 2024-03-24T03:14:35.956 INFO:teuthology.orchestra.run.smithi019.stderr: 96.4% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.e.log.gz 2024-03-24T03:14:36.064 INFO:teuthology.orchestra.run.smithi017.stderr: 89.9% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-24T03:14:36.065 INFO:teuthology.orchestra.run.smithi188.stderr: 92.0% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-24T03:14:36.221 INFO:teuthology.orchestra.run.smithi019.stderr: 92.1% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-24T03:14:36.650 INFO:teuthology.orchestra.run.smithi017.stderr: 90.5% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-24T03:14:37.535 INFO:teuthology.orchestra.run.smithi188.stderr: 94.8% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-mds.i.log.gz 2024-03-24T03:14:40.175 INFO:teuthology.orchestra.run.smithi019.stderr: 92.8% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.10.log.gz 2024-03-24T03:14:40.536 INFO:teuthology.orchestra.run.smithi188.stderr: 93.2% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.11.log.gz 2024-03-24T03:14:41.086 INFO:teuthology.orchestra.run.smithi017.stderr: 93.1% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-24T03:14:41.238 INFO:teuthology.orchestra.run.smithi188.stderr: 92.9% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.8.log.gz 2024-03-24T03:14:41.351 INFO:teuthology.orchestra.run.smithi019.stderr: 93.1% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-24T03:14:41.601 INFO:teuthology.orchestra.run.smithi188.stderr: 92.9% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-24T03:14:41.803 INFO:teuthology.orchestra.run.smithi019.stderr: 92.9% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-24T03:14:41.942 INFO:teuthology.orchestra.run.smithi017.stderr: 93.0% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-24T03:14:42.119 INFO:teuthology.orchestra.run.smithi017.stderr: 92.7% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-24T03:14:43.109 INFO:teuthology.orchestra.run.smithi019.stderr: 92.7% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-24T03:14:43.110 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-24T03:14:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m7.390s 2024-03-24T03:14:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m23.555s 2024-03-24T03:14:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m0.760s 2024-03-24T03:14:43.811 INFO:teuthology.orchestra.run.smithi017.stderr: 92.9% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.9.log.gz 2024-03-24T03:14:43.812 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-24T03:14:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:real 0m8.098s 2024-03-24T03:14:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:user 0m26.530s 2024-03-24T03:14:43.813 INFO:teuthology.orchestra.run.smithi017.stderr:sys 0m0.888s 2024-03-24T03:14:43.946 INFO:teuthology.orchestra.run.smithi188.stderr: 92.7% -- replaced with /var/log/ceph/2a682800-e989-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-24T03:14:43.948 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-03-24T03:14:43.948 INFO:teuthology.orchestra.run.smithi188.stderr:real 0m8.229s 2024-03-24T03:14:43.949 INFO:teuthology.orchestra.run.smithi188.stderr:user 0m25.756s 2024-03-24T03:14:43.949 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m0.889s 2024-03-24T03:14:43.949 INFO:tasks.cephadm:Archiving logs... 2024-03-24T03:14:43.949 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480/remote/smithi017/log 2024-03-24T03:14:43.950 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-24T03:14:47.817 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480/remote/smithi019/log 2024-03-24T03:14:47.818 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-24T03:14:51.441 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480/remote/smithi188/log 2024-03-24T03:14:51.442 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-24T03:14:55.205 INFO:tasks.cephadm:Removing cluster... 2024-03-24T03:14:55.206 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a682800-e989-11ee-95cd-87774f69a715 --force 2024-03-24T03:14:55.404 INFO:teuthology.orchestra.run.smithi017.stdout:Deleting cluster with fsid: 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T03:14:55.698 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a682800-e989-11ee-95cd-87774f69a715 --force 2024-03-24T03:14:55.900 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T03:14:56.306 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a682800-e989-11ee-95cd-87774f69a715 --force 2024-03-24T03:14:56.506 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: 2a682800-e989-11ee-95cd-87774f69a715 2024-03-24T03:14:56.795 INFO:tasks.cephadm:Removing cephadm ... 2024-03-24T03:14:56.795 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-24T03:14:56.813 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-24T03:14:56.830 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-24T03:14:56.845 INFO:tasks.cephadm:Teardown complete 2024-03-24T03:14:56.845 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-24T03:14:56.856 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-24T03:14:56.856 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-24T03:14:56.859 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-24T03:14:56.873 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-24T03:14:56.986 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-24T03:14:56.995 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-24T03:14:56.996 DEBUG:teuthology.orchestra.run.smithi017:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T03:14:56.998 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T03:14:57.000 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T03:14:57.012 INFO:teuthology.orchestra.run.smithi017.stderr:bash: line 1: ntpq: command not found 2024-03-24T03:14:57.013 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-03-24T03:14:57.014 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-03-24T03:14:57.018 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T03:14:57.018 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2024-03-24T03:14:57.018 INFO:teuthology.orchestra.run.smithi017.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +1135us[+1135us] +/- 82ms 2024-03-24T03:14:57.018 INFO:teuthology.orchestra.run.smithi017.stdout:^* hv02.front.sepia.ceph.com 3 7 377 101 -854us[ -850us] +/- 64ms 2024-03-24T03:14:57.018 INFO:teuthology.orchestra.run.smithi017.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +234us[ +234us] +/- 75ms 2024-03-24T03:14:57.018 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T03:14:57.020 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T03:14:57.020 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-03-24T03:14:57.020 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 37 +1077us[+1065us] +/- 82ms 2024-03-24T03:14:57.020 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -913us[ -926us] +/- 64ms 2024-03-24T03:14:57.020 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +168us[ +168us] +/- 75ms 2024-03-24T03:14:57.020 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T03:14:57.020 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T03:14:57.020 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-03-24T03:14:57.020 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +1079us[+1079us] +/- 82ms 2024-03-24T03:14:57.020 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -899us[ -898us] +/- 64ms 2024-03-24T03:14:57.020 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +177us[ +177us] +/- 75ms 2024-03-24T03:14:57.021 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T03:14:57.021 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-24T03:14:57.030 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-24T03:14:57.030 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-24T03:14:57.083 DEBUG:teuthology.orchestra.run.smithi017:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-24T03:14:57.170 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-24T03:14:57.252 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-24T03:14:57.322 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T03:14:57.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T03:14:57.353 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T03:14:57.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T03:14:57.388 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T03:14:57.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T03:14:57.419 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-24T03:14:57.430 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-24T03:14:57.472 INFO:teuthology.task.internal:Duration was 1943.064032 seconds 2024-03-24T03:14:57.473 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-24T03:14:57.510 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-24T03:14:57.511 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-24T03:14:57.513 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-24T03:14:57.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-24T03:14:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T03:14:57.558 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T03:14:57.565 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T03:14:57.992 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-24T03:14:57.992 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi017.front.sepia.ceph.com 2024-03-24T03:14:57.993 DEBUG:teuthology.orchestra.run.smithi017:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-24T03:14:58.053 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-03-24T03:14:58.053 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-24T03:14:58.082 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-03-24T03:14:58.083 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-24T03:14:58.104 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-24T03:14:58.104 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-24T03:14:58.106 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-24T03:14:58.125 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-24T03:14:58.172 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-24T03:14:58.173 DEBUG:teuthology.orchestra.run.smithi017:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-24T03:14:58.175 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-24T03:14:58.200 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-24T03:14:58.578 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-24T03:14:58.589 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-24T03:14:58.590 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-24T03:14:58.616 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-24T03:14:58.647 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-24T03:14:58.675 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-24T03:14:58.685 DEBUG:teuthology.orchestra.run.smithi017:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-24T03:14:58.687 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-24T03:14:58.690 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-24T03:14:58.710 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = core 2024-03-24T03:14:58.717 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-03-24T03:14:58.740 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-03-24T03:14:58.766 DEBUG:teuthology.orchestra.run.smithi017:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-24T03:14:58.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T03:14:58.780 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-24T03:14:58.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T03:14:58.802 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-24T03:14:58.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T03:14:58.820 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-24T03:14:58.831 INFO:teuthology.task.internal:Transferring archived files... 2024-03-24T03:14:58.832 DEBUG:teuthology.misc:Transferring archived files from smithi017:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480/remote/smithi017 2024-03-24T03:14:58.832 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-24T03:14:58.875 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480/remote/smithi019 2024-03-24T03:14:58.876 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-24T03:14:58.920 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480/remote/smithi188 2024-03-24T03:14:58.921 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-24T03:14:58.959 INFO:teuthology.task.internal:Removing archive directory... 2024-03-24T03:14:58.959 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-24T03:14:58.961 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-24T03:14:58.964 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-24T03:14:59.007 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-24T03:14:59.019 INFO:teuthology.task.internal:Not uploading archives. 2024-03-24T03:14:59.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-24T03:14:59.065 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-24T03:14:59.065 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-24T03:14:59.067 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-24T03:14:59.070 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-24T03:14:59.079 INFO:teuthology.orchestra.run.smithi017.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 24 03:14 /home/ubuntu/cephtest 2024-03-24T03:14:59.082 INFO:teuthology.orchestra.run.smithi019.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 24 03:14 /home/ubuntu/cephtest 2024-03-24T03:14:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 24 03:14 /home/ubuntu/cephtest 2024-03-24T03:14:59.085 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-24T03:14:59.122 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-24T03:14:59.185 INFO:teuthology.nuke:Checking targets against current locks 2024-03-24T03:14:59.214 DEBUG:teuthology.nuke:shortname: smithi017 2024-03-24T03:14:59.214 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T03:14:59.241 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 02:33:08.398239', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFUe7d7XjiupzXqAE5HnT+Gg5TRAtRnQbRE2JxfXVp+r309vjJvzhPnBsire81OyaoHVhH6cONnoY0+1PaOkIMs='} 2024-03-24T03:14:59.245 DEBUG:teuthology.nuke:shortname: smithi019 2024-03-24T03:14:59.246 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T03:14:59.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 02:33:08.396749', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIFq+/J9qEtq5l/LPnU8IS5rsliRZ1DBx8n7RblI04WyROxl+uPuIOsSgWhkwuIg2GFRANkhCEJd+GTScji7TUQ='} 2024-03-24T03:14:59.272 DEBUG:teuthology.nuke:shortname: smithi188 2024-03-24T03:14:59.273 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T03:14:59.291 INFO:teuthology.orchestra.console.smithi017:Power off 2024-03-24T03:14:59.292 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-24T03:14:59.301 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 02:33:08.399343', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBon3nCu8YzMfhuhPKyUB3z6c+xnfjg13+X6+0I//2ZJIam9fr4+9G/H+hKTQnfyIv2KEuwGI1oHWXJeksZa2wg='} 2024-03-24T03:14:59.315 DEBUG:teuthology.orchestra.console.smithi017:power off output: Chassis Power Control: Down/Off 2024-03-24T03:14:59.316 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T03:14:59.329 INFO:teuthology.orchestra.console.smithi019:Power off 2024-03-24T03:14:59.329 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-24T03:14:59.341 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-03-24T03:14:59.350 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2024-03-24T03:14:59.350 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T03:14:59.366 INFO:teuthology.orchestra.console.smithi188:Power off 2024-03-24T03:14:59.366 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-24T03:14:59.375 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-24T03:14:59.388 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2024-03-24T03:14:59.388 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T03:14:59.412 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-03-24T03:15:03.341 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T03:15:03.376 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T03:15:03.413 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T03:15:03.452 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-03-24T03:15:03.485 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-24T03:15:03.521 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-03-24T03:15:07.456 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T03:15:07.486 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T03:15:07.522 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T03:15:07.565 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off 2024-03-24T03:15:07.594 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-24T03:15:07.631 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-03-24T03:15:07.666 INFO:teuthology.orchestra.console.smithi017:Power off completed 2024-03-24T03:15:11.599 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T03:15:11.631 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T03:15:11.710 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-03-24T03:15:11.740 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2024-03-24T03:15:11.811 INFO:teuthology.orchestra.console.smithi019:Power off completed 2024-03-24T03:15:11.841 INFO:teuthology.orchestra.console.smithi188:Power off completed 2024-03-24T03:15:11.946 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} duration: 1943.0640320777893 failure_reason: '"2024-03-24T02:52:29.811398+0000 mon.a (mon.0) 189 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: mchangir success: false 2024-03-24T03:15:11.946 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-24T03:15:12.032 INFO:teuthology.run:FAIL